Instrument Development Overview
The first, and most important, part of a needs analysis is the development of an instrument(s) to gather information from the groups that will be affected by the provided solution. These instruments should provide quantitative, as well as qualitative data. For reliability purposes, the instrument should be applied to 7-20 representatives from each affected group. In developing the instrument(s), you should first consider who your user groups are and decide what type of information you will need to gather from them. Information requested may include their technology proficiency and comfort, specifications about their technical environment, etc. Also, if there are key players involved in the project, a well designed interview would be an effective way to gather information. These people may include the Subject Matter Expert(SME), clients, tech department staff, etc. Be ready to defend your choices of instruments and questions, as well as the methodology of collecting the data.
Identifing Audiences
Think about your problem? Who are the people currently impacted by the problem? Who are the people possibly impacted by the solution? What groups of people may be able to shed more light on the problem OR solution?
These are important questions in identifying an audience to assess. Before you can start to develop assessment tools, you need to have a complete picture of ALL of the stakeholder AND how they relate to each other and the problem/solution.
Additionally, you should make some assumptions about their knowledge in regards to the problem/solution. These assumptions may includes terminology, willingness to participate, level of involvement, etc.
Identifying Objectives
The big question here is "What do you want the survey to accomplish?". Instrument development begins identifying goals/questions that you want the instrument to answer about the problem and any possible proposed solutions.
Instrument Development
Instruments should be develped with the goal of all respondents completing the survey completely and accurately. For this reason, there is a general outline used in developing surveys and interviews:
- Start with an introduction that explains the purpose. This introduction should capture the responders interest and let them know why their group has been chosen to respond.
- The instruments should follow a logical order with major topic areas broken up with headers. Additionally, the questions should increase in difficulty. Starting with simple questions like name, age, etc.
- Instruements should use both quantitative and qualtitative questions to provide users with a variety of ways to respond. Often times a quantitative question can be a lead in or opening for a follow up qualitative question. Using this strategy will likely give you more focused responsed on qualitative questions.
- The survey should conclude with less important issues and some type of wrap up thanking the user for their time.
Question Development
The most important part of developing an instrument is developing the questions to be effective and easy to understand. Use the following set of bullet points to help develop your questions:
- Make questions easy to understand
- Do not write vague questions or questions that can be misinterpreted
- Write valid questions. A valid question is a question that provides useful information. Asking a user if they like something is not the same as asking them how often they use it.
- Don't ask two questions in one.
- Don't inject your views into questions or you may bias the answers. Keeep question neutral.
- Limit the number of true/false questions as they do not always provide complete information. Optionally, you can follow them up with a way for the user to expand.
- Be careful in likert scale type questions to avoid giving the user a neutral answer. Forcing them to pick a side will provide better information. Same is true for rating and ranking.
- Test your questions on people not in the focus group and not on your team. Ask them for feedback as to any necessary clarifications on questions.