NEEDS ASSESSMENT AND DISCREPANCY EVALUATION

Educational interventions result from perceived needs in an educational setting. Perceptions of need may come from many sources and happen at many levels. For example, the teacher may feel that Johnny is not doing well in his second-grade reading and switch him to another group, or provide some special remedial activities. Or, a school system may be moved to action by the poor performance of tenth-graders on a standardized achievement test and implement a special tutoring program.

Some of the ways educational needs are defined:

Discrepancy Analysis: needs are defined as the difference between expected or desired levels of performance and actual performance.

Democratic View: needs are changes desired by a group or groups of stakeholders in the school system

Diagnostic View: need is a lack or deficiency that is hampering or harming the learning process of students.

(1) defining current expectations
(2) determining the depth and scope of any discrepancy between performance and expectations
(3) using discrepancy information to decide on changes to expectations or to teaching/learning strategies.

The advantage of the discrepancy evaluation approach is the ease and clarity by which it can be used by persons without extensive training in evaluation methodology. A discrepancy is the difference between "what is" and "what should be." That difference can be stated as a measurable objective. For instance, if 30% of the students in an average class are at or above the 50th percentile in a standardized competency test, the discrepancy is obvious. In any average group of 30 or more students, at least 50% of the class should score above the 50th percentile on a standardized competency test. The discrepancy in this example is that 70% of seventh-graders scored below the 50th percentile. Ideally, only 50 should score below the 50th percentile. This 20% discrepancy easily translates into a measurable objective for change. Defining a discrepancy using the discrepancy evaluation approach requires a needs assessment and data gathering.

Goals can also be used as a means of developing discrepancy statements. A goal is a desired general outcome. A school district might include among its goals the expectations for all elementary school pupils in the district to:

• Develop skills in reading, writing, speaking, and listening.
• Develop pride in work and a feeling of self-worth.
• Develop good character and self-respect.
• Learn how to examine and use information.
• Develop a desire for learning now and in the future.
• Gain a general education in the use of numbers, mathematics, and computation.

• Discrepancy evaluation is a method of ongoing educational evaluation based upon perceived differences between "what is" and "what should be."
• The purpose of discrepancy evaluation is to determine the effect of the education program on student performance and to set any necessary plans for program change.
• A discrepancy evaluation is a concise, data-based definition of the difference between "what is" and "what should be."

NEEDS ASSESSMENT INSTRUMENTS

Researchers employ a variety of techniques in assessing needs. The most common needs assessment instruments are (1) surveys, (2) interviews, and (3) attitudinal surveys. Some of the instruments are very complex while others are easy to administer and score. The type and the quality of instrument used in the needs assessment stage of a research project or a GTEP practicum will greatly influence the intervention and the results. There are also some basic procedures that should be followed in writing any form of a needs assessment instrument.

Criteria for Survey Writing

The instrument must have an appropriate title. The title should describe the topic of the survey and mention the target audience, at a minimum.

Examples

:Kindergarten Students' Parent-Involvement Survey

Staff Development Questionnaire for Middle-Grades Faculty

School Subject Interest Survey for Third-Grade Students

Every survey, test, or questionnaire must contain a statement about the purpose of the instrument and explicit directions for completion. The more explicit the directions are, the more chance there is that the population being surveyed will provide data that will be easier to interpret.

Examples:

Writing items for needs assessment instruments, tests, or for any other purpose requires a great amount of knowledge, skill, and practice. Items should always be written with great care and much planning. It is a good idea to submit a survey or a questionnaire for review to a colleague or a content expert prior to implementation. It is desirable to submit the questionnaire or survey to a panel of experts to insure validity.

Common errors can be avoided by asking yourself the following questions during the writing process.

1. Is the question related to the research problem and objective?
2. Is the type of question appropriate?
3. Is the item clear and unambiguous?
4. Is the question in any way leading?
5. Does the question demand knowledge and information that the respondents do not have?
6. Does the question demand personal or sensitive information?
7. Is the question in any way biased?
8. Are there enough questions to adequately explore the issue

Surveys

Surveys are widely used by researchers in education. Surveys can be adapted to measure human behavior and attitudes, as well as physical concepts. Results of surveys are used to assess goals, establish baseline data, and analyze trends across time, just to name a few. Well constructed surveys have some characteristics in common. They are systematic and executed to ensure appropriate coverage and efficient data collection. It is important for a survey to be representative. It should reflect the population either by including everybody or everything, or by using a scientific sampling procedure. Constructing an objective survey is important because it allows the data to be as observable and as explicit as possible. Finally, a well constructed survey should yield quantifiable data that can be expressed in numerical terms.

Most surveys are dependent on direct communication with persons having characteristics, behaviors, attitudes, and other relevant information appropriate for a specific investigation. This makes them reactive in nature, which means that they elicit a reaction from a respondent. Although direct communication is often the most efficient way of collecting data, it does present some problems. Among the limitations are:

1. Surveys tap only respondents who are accessible and cooperative.
2. Surveys produce "response sets" or proneness to agree with positive statements or questions.
3. Surveys are vulnerable to bias, a tendency for respondents to constantly give very high or very low ratings.
4. Surveys produce the "Hawthorne Effect" among respondents by making them feel special as participants in research, which in turn effects their responses.

Some of the general guidelines for designing surveys are the following:

1. Define the purpose and scope of the survey prior to writing any items.
2. Use closed-ended questions as opposed to open-ended questions.
3. Avoid using existing surveys.
4. Be aware of potential bias in the items.
5. Use standard English and appropriate terminology.
6. Analyze the comprehensiveness of the instrument prior to administration.
7. Field test the survey whenever possible, or have a peer review.
8. Ensure easy scoring.

There are several types of surveys. When choosing a particular one, the researcher should keep in mind the appropriateness of the instrument for the project.

1. Survey of Records. These surveys are nonreactive because they do not involve responses from people. These surveys are usually inexpensive and provide excellent baseline data if the records are accurate and current. Record surveys can also be incomplete, inaccurate, and misleading, depending on how the records were compiled.

2. Mailed Questionnaires. This is the most commonly used method and the one that can be most misleading. It is inexpensive, self-administering, anonymous, and comprehensive. Some of the disadvantages can be low response rate, no assurance that the questions were understood, or that the intended respondent answered the questions.

3. Telephone Surveys. These are widely used in place of face-to-face interviews. The advantages are that they provide unlimited call-backs, are not restricted by time, have extensive geographical coverage, and allow the respondents to be more candid. Drawbacks might include: telephone interviews are sometimes viewed as intrusive, problems occur when some individuals have unlisted phone numbers or have no phones. There is no opportunity to read body language.

Interviews

Individual Interviews are especially useful for in-depth exploration of problems and sensitive issues. This form of inquiry is also useful in collecting information when existing data are insufficient. Face-to-face interviews are probably the most expensive to conduct. They are flexible and adaptable and allow for interpretation of body language. Researchers should be well trained in interviewing techniques so as to be able to elicit valid responses. An interview may be skewed because of personality differences between the interviewer and the client. The findings are often difficult to summarize because of lack of objectivity. Group interviews are used in group settings because they are efficient and provide for brainstorming, which increases interaction and reflects group consensus. This form of survey can also foster conformity, suppress individual differences, and polarize opinions.

Research interviews can be unstructured, semistructured, or structured. An unstructured interview gives the respondent broad freedom to express himself in his own time and in his own way. Frequently the information is of a highly personal nature and may even be threatening. Of the three interview formats, this one is the most vulnerable to biases and error, usually due to the inexperience of the interviewer. An unstructured interview is commonly used in client-centered therapy.

Semistructured interviews are constructed on a few essential questions, which the interviewee uses to expand and explore in other directions. This structure also allows for additional in-depth exploration of issues and problems not included in more structured experiences. In both the unstructured and semistructured interviews, the interviewers require training.

Structured interviews are well defined and often resemble an objective questionnaire. There is very little room for exploration and digression. They are factually oriented and aimed at obtaining specific information. Structured interviews are especially suitable when information is required from all respondents and when the type of information fits the format. The interviewers do not require specialized training to conduct structured interviews.

Students often use interviews to collect information for a project. It is therefore important to address a few principles and guidelines regarding this method of inquiry. The first step is to plan an interview in order to ensure that it will yield the desired information. The planning steps should include the following.

1. Define the reason for using the interview as opposed to another data collection technique.
2. Translate the general goals of the study or project into objectives that can be fitted to the particular interview pattern.
3. Develop a tentative set of procedures to use during the interview.
4. Develop a satisfactory method of coding and decoding responses.

1. Questions should be stated in language familiar to the respondents. Avoid ambiguity.
2. Avoid leading or biased questions.
3. Ensure that the frame of reference of each question is clear, so that each respondent understands the same thing.
4. Ascertain whether the population being interviewed has the information sought by the interviewer.
5. Ensure that the respondents appreciate the purpose of each question.

An attitudinal scale is a device consisting of symbols that have a value assigned to them by the researcher. Attitudinal scales are very popular measurement instruments with researchers and GTEP students because they are easy to administer and yield quantifiable data. The best known type of a scale is Likert scale, which consists of items to which a subject responds with varying degrees of intensity. The rating scale usually consists of numbers or letters that signify a range of preference from the strongest to the weakest degree. Five to seven positions are recommended.

For example, commonly used alternatives are:

strongly agree (SA),
agree (A),
undecided (U),
disagree (D),
strongly disagree (SD),

 Numbers can also be used to represent values, with the lowest number 1 representing the low spectrum of the scale and 5 representing the highest number.

1 (poor), 2 (below average), 3 (average), 4 (very good), 5 (excellent)