8 things to keep in mind when designing a research study
Developing a high-quality online research study can be difficult. Gathering the best data for your research hinges on good study design. And there are many different factors to consider.
From the visuals to the question order, research design affects the quality of responses you'll get from participants. Not sure how to design a research study that will get the results you need? Don’t worry. In this guide, we’ll take you through 8 key things you need to keep in mind.
The questions you ask participants can affect the quality of their responses. You need to think carefully about two key things:
How you phrase your questions is important. Certain words have different meanings, and participants may interpret them in different ways. Your questions should also align with the response format. A “yes/no” question won’t work with a Likert scale, for example.
Keep the language simple and remove any generic expressions. You should also avoid double negatives, unless the question is reverse scored.
Ask general questions first to ease participants into the survey. You can then move on to more personal questions. Group topics together under specific headings to make them clear for participants.
You should always pilot test questions to ensure that participants can understand them. Test the question order as well to avoid data bias.
All research surveys are prone to bias. However, there are some simple ways to minimize the impact of bias on your findings. Types of bias to consider include:
The Likert scale is an ordinal scale. It measures participant responses with five to seven fixed options. For example, a Likert scale might have “Strongly Agree” and “Strongly Disagree” at opposite ends of the scale. There is usually a neutral option in between. If “yes/no” questions are too restrictive for the data you want to collect, using a Likert scale is a better option.
When you’re working with data, it’s important to be aware of proper data collection procedures. Ensure you’re keeping data safe by:
Before collecting data from participants, you should always get their consent through a consent form. This form should be clear and easy for participants to understand. It should explicitly state:
Always make sure you ask the participant if they understand the consent form. Then check if they are happy to proceed. You should also make them aware that they can remove their consent at any time.
They can then click a check box as evidence that they have consented. Your institution may have their own wording requirements for consent forms, so research this first.
Participants will want to know the details of your study. They might be curious about the purpose of the study. Or they might have questions about what they need to do.
You can give them this information in the study description. Share details like the study’s aim, what the participants will need to do, and how you’ll use the data gathered. For a full list of what to include, read our detailed guide to writing the study description and debrief.
For the debrief, always thank the participant for taking part - and make sure you inform them of any deception used in the study.
To ensure your study is ethical, you need to pay your participants fairly for their time. This can also improve the quality of the data they provide. At Prolific, we strongly believe in the importance of ethical research. That’s why we’ve implemented a minimum hourly reward of £6/$8.
If you want more participants to engage with your study and provide accurate answers, you may need to pay more. We recommend you pay at least £9/$12 per hour. But this will vary depending on different factors, such as the level of effort needed for the study.
The visual design of your research study can also impact the data quality. For example, if your font is hard to read, participants may misinterpret the question. Choose a simple font, such as Arial, in a readable size.
Pick colours carefully. There should be a clear contrast between the writing and the background. Use bright colours sparingly, especially for sensitive topics. For longer studies, it can also help to include a progress bar.
To get the best data for your research, you need to design your study carefully – and find trusted participants to take part. With Prolific, finding the right participants is simple. Our platform features 130,000+ vetted participants from diverse populations around the globe.
Want to learn more about effective research design? Find out how to design and launch a high-quality online study in The complete best practice guide to online research. Download your copy now.
Today Prolific is turning 5 years old – Happy Birthday to us! 🥳 It's been a remarkable journey so far. 3000+ researchers from science and industry have used Prolific last year, we have 45,000 quarterly active participants, and we've seen 200% year-on-year growth. But we're only getting started. In this post, I'll tell you a little bit about our journey, give credit where it's due!, and tell you about our exciting plans for the future.
Fresh out of YC's Summer 2019 batch, we want to share some of our most interesting learnings. If you're a startup founder or enthusiast and want to learn about product-market fit, growth experimentation and culture setting, you're in the right place!