How to evaluate the project?

Evaluating the Project 

The project should be evaluated at two levels. The first of these levels is the text and the logic of the presentation. The second level of evaluation should focus on the student’s mastery of the material. There are certain minimum requirements that the project or thesis must meet before it should be considered for acceptance. These minimal requirements deal with the manner of presentation of the project problem, its parameters, and analysis. To facilitate the examination of a project, I’ve provided a comprehensive checklist and inventory of the minimal requirements.

For example:

1. Is the topic area of the study clearly specified early in the text (within the first two pages)? 
2. Is the topic area of the project derived from or cast into its historical perspective? 
3. Are the major independent and dependent variables logically derived and their importance clearly evident? 
4. Is the specific problem that the study addresses clearly stated early in the text (within the first ten pages)? 
5. Is the problem an important one and is its importance clearly specified? 
6. Are the parameters of the study (limitations and delimitations) specified early in the presentation (within the first fifteen pages)? 
7. Are the objectives of the research clearly stated? 
8. Is a rationale presented for the objectives? 
9. Are the important terms in the study defined? 
10. Is the related literature summarized? 
11. Is the relevance of the literature to the present study clearly specified? 
12. Is the related literature critique (an examination of strong points, weak points, problems of methodology, etc.)? 
13. Are the hypotheses of the study clearly and correctly stated? 
14. Is it evident that the hypotheses flow from the problem, objectives, and related literature? 

1. Are the research participants in the study described? 
2. Is the sample size the study presented and justified (is the N adequate)? 
3. Are the subgroups, if any, used in the study described and is there a clear presentation of their comparability? 
4. Are the levels of the independent variables clearly defined and/or operationalized? 
5. Are all the research instruments used in the study presented? 
6. Is there an examination of each instrument’s reliability and validity? 
7. Is there a clear description of the procedures of the study in sufficient detail that another researcher could replicate the process? 
8. Do the research instruments assess the dependent variables? 
9. Is the design of the study described, and, for more elaborate designs, schematically presented?
10. Are appropriate descriptive statistics (for example, means and standard deviations) presented in the analysis of results?
11. Are the inferential statistical procedures appropriate for the design? 
12. Are all of the statistical procedures specified in sufficient detail that the reader can replicate the analysis?
13. Are the results of the inferential statistical analysis (specific values of statistical test results) presented with degrees of freedom, if necessary, and levels of confidence? 
14. Are the findings relative to each hypothesis summarized and a decision (rejection or failure to reject) made about the hypothesis?
15. Is there an interpretation of what of findings mean? 
16. Are the findings contrasted to other related research findings? 
17. Is there a thorough discussion of the problems encountered in the study. 
18. Is there a presentation of the practical implications that the findings may have? 
19. Is there an examination of rot potential research studies which grow out of the present study? 
20. Is there a logical flow from the statement of the hypotheses and research procedures, to the finding and interpretations of the study? 

To help insure a degree of quality control with the final project product, I recommend that the student be required to complete the check list and discuss the ratings with his/her chairperson before a committee is made to schedule the final oral examination, I suggest that the student be required to underline the parts of the project text which address each of the 35 points. Additionally, I recommend that the student be required to complete a “pre-oral examination” during this period. 

Relative to the methods section of the Oral Examination, the student should be able to:

1. discuss the possible challenges to the internal and external validity of his/her research design; 
2. discuss the appropriateness of the methods used in validating the instrumentation; 
3. critically examine the methods used to establish reliability; 
4. examine other variables which were not controlled and which may have had a significant confounding influence on the research model; 
5. critically analyze the sample size used in the design as well as its representative ness; 
6. discuss the appropriateness of, and problems of implementing, the research. 

Concerning the analysis procedures, the student should be able to: 

1. discuss the appropriateness of the statistical procedures used in the design; 
2. compare the statistical model with other models, pointing out the advantages and disadvantages of his/her statistical model;
3. demonstrate how the statistics were calculated and discuss principle terms, line interaction, eigne values, factor loadings, etc; and 4) discuss the statistics which relate to each hypothesis. 

Finally, concerning the interpretation section of the Oral Examination, the student should be able to:

1. discuss the findings of the study relative to the literature; 
2. discuss possible alternative findings if changes had been made in the levels of the independent variables or procedures; 
3. discuss the practical implications that the study may have;
4. discuss implications for further research; and 5) defend his/her interpretations.