Designing a Research Study

Project task list

1. Design, planning, and recruitment
2. Survey administration, data collection, and cleaning.
3. Analysis and report generation
4. Results dissemination to the sponsor, participating institutions, and the general public

Phase 1: Design, planning, and recruitment

Construct project communications:

• Collect sample communication used for previous student course material affordability projects.
• Review and revise, and update communications.
• Review proposed communications with any advisory board, sponsors, and potentially participating institutions.

Recruit Participating Institutions:

• Use communications to recruit potential partner institutions to the project. Determine any IRB requirements they may have. Determine any special requirements they might have for the survey instrument or the survey distribution.
• Determine the timing for data collection for each participating institution (survey open date and survey close date). Create list of participating institutions with key contacts (name, title, email, and phone number).

Prepare questionnaire:

• Review questionnaires used in previous projects (e.g., VIVA Virginia, Pennsylvania, Illinois, Kentucky, Florida, and Oregon). Build on the other projects to draft an initial version of the questionnaire.
• Review the questionnaire; circulate internally for feedback.
• Receive feedback on the questionnaire.
• Create a final version of the questionnaire based on feedback.
• Program questionnaire into online survey system – typically add timing measures for all questions for use in testing. Test overall flow, wording, and definitions with an outside test group.
• Test all possible skip options in the survey. Test that only values in the appropriate range can be entered (e.g., years, counts, etc.). Test that open-ended responses correctly accept special characters. Test that questionnaires can be reentered correctly.

Phase 2: Survey administration, data collection, and cleaning

Create versions of the questionnaire for all partners:

• Determine all institutional-specific wording needed for each participating institution (e.g., local contact information, any IRB-required description, etc.) to include on the survey landing page.
• Revise each partner questionnaire to include any institutional-specific wording on the landing page.
• Provide a list of survey URLs, for each participating institution to be shared with partners.
• Provide each participating institution with their specific survey URL.

Set up a Student Incentive System

• Determine the number and type of incentives to include in a raffle to encourage student responses.
• Construct a raffle sign-up form separate from the main survey, so there is no linkage between survey responses, student names and email, and raffle winners.

Construct invitation and reminder emails.

• Create model survey invitation and reminder messages based on prior studies that participating institutions can adapt.
• Get feedback from partners on messaging – revise if required.

Administer the survey.

• Determine how each partner wishes to distribute the survey.
• Activate and test each partner survey. Monitor all open-ended questions for any comments on the questionnaire and/or the survey process.
• Monitor data collected daily for any issues. Create a weekly tally of the number of responses by institution.
• Create a weekly summary of top-line data from completed surveys.

Award Student incentives

• Remove duplicate student responses from the analysis data set and raffle pool.
• Randomly select raffle winners from among all qualified respondents.
• Notify all student raffle winners.
• Monitor all award notifications for bounces or blocked messages. Select alternative award winners to replace all unreachable winners.

Phase 3: Analysis and report generation

Create the analysis database.

• Download all data on weekly basis. Clean all data and create a single analysis database in a statical analysis program, such as SPSS. Add all variable definitions and labels to the analysis file. Merge in all other information known about the institutions from Federal data (e.g., IPEDS), membership data or other partner data.
• Examine all incomplete surveys and determine whether they should be included. Check for duplicate responses, merge responses where appropriate. Recode selected variables. Create of indices using multiple variables.
• Create a full set of frequency tables for all variables for the entire sample.
• Produce a comprehensive set of reference tables (all the important variables by all the meaningful independent factors) for the full sample.
• Produce a set of reference tables that mirror those for the entire sample for each participating institution with a sufficient number of student responses.

Draft report.

• Create an initial outline and draft of each section of the report.
• Create additional analysis tables for each section as needed.
• Circulate the draft report to the broader review group for review and comment.

Proofreading, copy editing, common style.

• Reviewed and quality check draft report by editors and proofreaders.
• Fact-check all data in tables and charts.
• Draft methods description for inclusion in report.

Create institutional-specific tables and charts.

• Create a draft report format that provides tables and charts that present the results for a specific institution compared to the overall results for the state and (if desired) for the national sample.
• Provide feedback on the draft institutional report.
• Create institutional-specific reports for all institutions with sufficient student responses.
• Distribute institutional-specific reports to all institutions with sufficient student responses containing all aggregated data tables and charts.

Phase 4: Results dissemination

Create a distribution and publicity plan.

• Determine the process for release of the primary report, and distribution of institutional reports.
• Draft press release (if needed) and schedule any presentations of the findings.