Computer-Assisted Assessment (CAA)

Core Edition of National Survey Questionnaire 2004

Thank you for taking the time to fill in this questionnaire.
This is an abridged version of the full questionnaire. Some questions appear in an abbreviated form, whilst questions 5, 11, 20 and 26 are skipped entirely.

All responses will be kept confidential, and the raw data deleted as soon as it has been analysed. If you have any queries about the questions, please contact us on 023 8059 2326 or email wiw@soton.ac.uk.
If you submit a completed questionnaires before the end of August 2004 we will enter your e-mail address into a draw for two £25 gift vouchers.


Personal and Professional Details (optional)

Email address

Job title

Subject specialism, if relevant

(a) Which of the following describe your position? (Please select all that apply)
      Part-time           Full-time

(b) Role
      Academic         Pedagogic support         ICT support

(c) Contract type
      Permanent        Short-term contract        Hourly paid

(d) Gender
      Male         Female

(e) Age   
      21-30       31-40      41-50      51-60      61-70+

(f) Highest qualification held   


1. Which of the following statements most accurately describes your experience with CAA?
     a) I currently use CAA, advise on the use of CAA or support CAA
     b) do not use CAA, advise on the use of CAA or support CAA
     c) I have used, advised on, or supported CAA in the past, but no longer do so.

2. Do you intend to use CAA in the future?    Yes   No

3. If you teach, are you currently using computer-aided learning (CAL) in your teaching, excluding word processing and spreadsheets?
    Yes   No

4. Does your institution provide CAA support for academics?  Yes   No

(a) If so, where is this located? Please select all that apply.
     As a central unit   Within faculties   Within academic departments

     Elsewhere (please specify)

(b) Does your institution have a dedicated CAA officer(s)?
     Yes     No

(c) Does your institution have a dedicated CAA department?
     Yes     No

6. In your view, does CAA save academics’ time?  Yes   No

     a) If 'yes', has this meant a significant shift in the workloads of other staff?
         Yes     No

7. a) Please list any advantages to using CAA that you have identified
   

7. b) Please list any disadvantages to using CAA that you have identified
   

8. What do you see as the critical success factors for the implementation of CAA?
    
 a) At the level of the individual academic
   
   
  b) At the level of the institution  
                 
   

9. What do you see as the main obstacles to the successful implementation of CAA?
     a) At the level of the individual academic
   
     b) At the level of the institution     
              

10. What do you perceive to be the attitude of each of the following groups towards CAA?

a) Students very negative negative neutral positive very positive
b) Academic staff very negative negative neutral positive very positive
c) Senior management very negative negative neutral positive very positive
d) External examiners very negative negative neutral positive very positive
e) Technical staff very negative negative neutral positive very positive


12. What future developments in CAA would you like to see?
    

13. If you CURRENTLY USE CAA, please complete the rest of the questionnaire.
IF YOU DO NOT, please scroll down to Q.23 and the Survey Feedback Details section at the bottom.

14. a) When did you start using CAA?

      Before 1985 - 1989      1990 - 1994      1995 - 1999      2000 - present

15. a) Do you use question banks?     Yes       No

16. a) Would you be interested in sharing questions with academics at other institutions?
          Yes      No

      b) If 'yes', can you give any details?
     

19. Please state the approximate number of computer-assisted assessments which you have delivered this year.

     
0-5       6-10       11-20       21-30       31-50       51-100       >100


      a) If you have delivered computer-assisted assessments this year, please say briefly what the subjects were
     

     b) How many assessments were at the level of:
        
         (i) Undergraduate Year 1?
        (ii) Undergraduate Year 2?
       (iii) Undergraduate Year 3 (or above)
       (iv) Postgraduate level?

     c) How many assessments were:

         (i) Diagnostic - to identify pre-course knowledge?
        (ii) Formative - to assist in the learning process?
       (iii) Summative - to assess post-course knowledge?
       (iv) Other - please specify

     d) How many assessments were invigilated?

     e) Which question types have you used this year?

          Multiple-choice
          Multiple-response
          Text input
          Numeric input
          Graphical hot spots
          Audio
          Video
          Other
    
     If other question types were used, could you say what kind?

     f) Please indicate the typical class size for CAA assessments?

     g) What other kinds of assessment are used in the modules?

         Essay – coursework
         Essay – exam
         Short answer questions
         Lab report
         Presentation
         Objective test
         Worksheets
         Other

         If Other, then please specify what

         (h) Please indicate the typical weighting (%) of the CAA within the modules? %

21. What CAA software packages are you using?
     Please list the packages and indicate which delivery method they relate to (summative/formative etc)
      


22. What do you see as the particular advantages and disadvantages of the CAA software you use?
      

23. Which of the following statements describe the experience of the staff at your institution with CAA?
         A significant percentage (20% or more) currently uses CAA
         Only a small group of enthusiasts use CAA
         No-one uses CAA
         Some have used CAA in the past, but no longer do so
         Some may be using it, but I am not aware of their use

24. a) Does your institution already provide an institution-wide Virtual Learning Environment (VLE) such as Blackboard or WebCT?
           No        Blackboard          WebCT        Other VLE

      b) If so, do you think it has:
          Encouraged people to use CAA
          Discouraged people from using CAA
          Made no discernable difference to the uptake of CAA

      c) Would you like to be able to export quizzes from your VLE to a CAA system?
          Yes No

      d) What impact is the VLE making on the way you use CAA?

        

25. Interoperability standards:
      IMS Question and Test Interoperability (QTI) is one specification for a standard way of sharing assessment data.
     
IMS QTI is designed to make it easier to transfer information such as questions, tests and results between different software applications.

     a) Are you already aware of CAA interoperability standards such as the IMS QTI specification?

         Very aware
         Moderately aware
         Not really aware

     b) How important to you is the ability to ‘future proof’ CAA interoperability,
        
such as the ability to transfer questions and assessments from one CAA system to another?
        Very important
        Moderately important
        Not really important
        Not at all important

     c) What impact is interoperability making on the way you use CAA?
       

This is the end of the 2004 national CAA Survey. We would like to thank you giving your time and effort. As a small token of our appreciation we will enter your email address into a draw for two £25 Gift Vouchers. It is intended that the draw will take place at ALT-C in September 2004.

The results of this survey will be published in a variety of formats, including electronic publications available from the TOIA website.

Would you like to receive further information about results of the survey, other project activities and CAA developments?

      Yes     No

If there are any aspects of CAA you are particularly interested in, please list them below

Can we contact you by telephone or email to follow up on your responses?

      Yes       No

If so, please indicate suitable days and times to call

Telephone number

This is the end of the survey. Please press "Finish" below to submit your answers.