Implementing a Survey
When implementing a survey, there are several factors to consider. The first of which is the method of delivery. With the abundance of online survey tools available, online surveys are an easy option and also make it easy to send and receive the survey and results. That said, with the amount of email that people receive, not everyone will complete your survey. The average response for online surveys is about 30-40%. So, if you are looking for 100 responses, you will need to send the survey to at least 300 people. On the other hand, hard copy paper surveys, while harder to distribute and collect, can yield return rates of up to 60%. In a school setting, you can often get a higher return by isolating classes and having the students take the survey while in class and then collecting them when they are finished.
This bring us to our second point and that is the sample. It is important when performing a survey to get a cross-sectional representative random sample. Poor planning on sample audiences can lead to distorted or biased outcomes. For example, if we were surveying students about the possible level of interest in the development of a math club, what to you think the results would be if we surveyed a class of high level calculus students vs. a class of low level math students? What if our sample classroom was an AP Photography or Physics class? It is important to make the sample as random as possible in order to ensure unbiased results. Random sampling can be done by selecting a teacher at random and then selecting 5 random students in the class (maybe the students sitting in the four corners and the center, or the first five students alphabetically, etc.).
Implementing an Interview
When implementing an interview, the best way, most often is face to face. If that is not possible, then a phone interview is the next best choice. In a worst case scenario, you could email the interview questions and get the responses back. The email scenario is the worst because it does not allow for interaction. What if one of the responses brings a new question to mind, or leads the discussion in another direction? That type of interaction is much more difficult by email.
Depending on the level of depth of the questions, it may be a good idea to forward the questions to the interviewee ahead of time so that they have time to reflect and consider all aspects of their answers.
Organizing Results
When a survey is returned, the data needs to be organized and tabulated. This is easy for quantitative data. If you were to ask 50 users their preference of browser, and 36 of them respond that Chrome is their primary browser, you know that 72% of respondents prefer Chrome. Tables and spreadsheets are the best way to organize this type of data as they will do many of the calculations for you.
Qualtitative data from open ended questions is a little more difficult. At first, you should record all answers from all respondents on each question. Then, you can group those responses according to theme or tone. Then, the results can be summarized using one or two quotes from each group of responses that are representative of the opinions in that group.
Both a full output of the results and a results summary should be included in a needs analysis. That way, readers can get a quick glimpse of the results in a summary, but have access to the raw data if necessary for a deeper look. Often, the summary is part of the main report while the raw data is contained in an appendix.
An example of data summary is contained in the Needs Analysis file, while the raw data can be viewed in the Raw Data file.
Interpreting Results
Once the instruments have been administered, the team will need to analyze and interpret the results. Included in this analysis would be:
- a clear description of the audience and the skills they possess that may influence effective use
- a clear picture of the environment in which the product will be used (internet connections, devices, browsers, etc)
- an explanation of how the results of the instruments will effect the design of the program
An example of the interpretation and implications can be found in the Needs Analysis file.