There are numerous definitions to evaluation.

By Caro:

Program evaluation procedures judgements regarding the degree to which desired outcomes have been achieved or can be achieved.

Suchman:

Evaluation is the determination of the results

Which are attained by some activity.

Scriven:

Evaluation is a methodological activity, which combines performance data with a goal scale.

Evaluation in a broad sense is a part of everyday life. All of us continually evaluate judgments about the full range of purposeful human activity. In social programming, crucial decisions both at level of polity and individual treatment are often based on informal evaluations. Recognition of the variability and fallibility of informal evaluation has stimulated interest in the formation of standardized procedures, which would yield evaluative judgments. Formal evaluation has become a widespread and diversified activity. And there are several approaches of formal evaluation.

Accreditation is the oldest and most firmly established approach to formal evaluation concerned with social programs. Particularly in education and health, accrediting agencies often examine and make public judgments regarding program quality. In some cases licensing is contingent on a favorable judgment by an accrediting agency. Accreditation methodology characteristically involves comparisons of program inputs with standard established by professionals. In educational accreditation, for example, physical characteristics of classrooms, ratio of library books to students, and the levels of education of faculty members are important dimensions. Information is usually obtained through site visits. Importantly, accreditation is more concerned with potential than actual achievement.

Program analysis represents a second evaluation tradition that is distinguished by its emphasis on program activities. It focuses upon actual operations. Actual performance is often compared to an externally derived standard of performance.

Cost analysis is still another distinct approach to evaluation. At its most basic level, cost analysis seeks to account for program expenditures.

Evaluation research may be considered an another tradition that is distinguished by its central concern for outcomes of treatment. It attempts to determine whether changes sought through an intervention actually come about. The purpose is to evaluate the impact of social interventions such as new teaching methods, innovations in parole, and a host of others. Many methods can be used in evaluation research.

Evaluation research is probably as old as social research itself. The distinction between basic research and evaluative research is one of purpose than in method. In evaluation research, we can use any quantitative and qualitative methods.

Evaluative research applies the scientific method to problems that have administrative consequences, whereas basic research is concerned with problems of theoretical significance. There are differences in diffusion and generality. Basic research has a great potential for generality but a limited potential for immediate utilization. By contrast, evaluative research has limited potential for generality, but great potential for immediate utilization.

In recent years the field of evaluation research become an increasingly popular and active research specialty. In part, the growth of evaluation research reflects social scientists’ desire to make a difference in the world. At the same time, we can’t discount the influence of (1) an increase in federal requirements that program evaluation must accompany the implementation of new programs and (2) the availability of research funds to fulfill those requirements. In any case, it seems clear that social scientists will be bringing their skills into real world than ever before.

Evaluation research may be concerned with stable and well-established programs or with new programs. Formative evaluation is designed to improve a program while it is still fluid. Summative evaluation is designed to appraise a product after it is well established.

Eyewashes – focus on surface appearance to make program look good.

Whitewashes – cover up program failures

Submarine – political use of ER to destroy a program

Posturing – ritual research having littered substance

Postponing – conduct research instead of acting.

The topics appropriate to evaluation research are limitless. The evaluation research is usually sponsored.

A careful formulation of the problem, including relevant measurement and criteria of success and failure, is essential in evaluation research. In particular, evaluators must carefully specify outcomes, measure experimental context, specify the interventions being studied (those who participated and not participated) and population targeted by the intervention, and decide whether to use existing measure s or devise new ones.

Successful administration of evaluation research depends on cooperation from agency administrators and practitioners charged with implementing action programs. Even though they often advocate extensive collaboration and communication with administrators, evaluative researchers typically insist that they hold ultimate responsibility for research design and execution. Administrative interference with what social scientists consider to be critical issues in the design and execution of research is seriously resented.

Whether or not researchers are agency employees, they are readily drawn into staff-management conflicts. A number of observers have noted that acceptance of evaluative research if often accompanied by suspicion of research at lower levels. Because evaluation is linked to top administrators and involves examination of the activities of staff subordinates, evaluators are sometimes suspected of being management spies. Staff practitioners are likely to conceal real or imagined shortcomings. Such steps may add greatly to the evaluator’s difficulties in obtaining valid data.

Research neutrality is also likely to pose a problem with practitioners who consider a strong value commitment to their programs important. Some practitioners complaint that the skepticism of evaluative researchers may dampen the creativity of a productive group.

Purely mechanical demands of data collection may also create burden. Practitioners typically, and perhaps correctly, consider themselves overburdened with record keeping. Characteristically, even when record keeping is emphasized, records are not sufficiently accurate or complete to satisfy research criteria.

Conflict between research and service goals may interfere with the collection of data. When a research design calls for action inconsistent with immediate service goals, practitioners may disregard research needs in favor of providing service.

Different conceptions of efficient use of time may lead to mutual annoyance. A professional evaluator, for example, is not accustomed to turning in daily time sheets. His failure to do so can be interpreted as a sign of indolence by an administrator concerned with time and cost factors.

Status ambiguities may further strain relationships. If an evaluator has more formal education but less clinical experience than his administrator and practitioners’ counterparts, conflicts may result. Sometimes practitioners have claimed that evaluators are not incompetent because they do not understand the practical problems of an action agency.

The publication of the result of evaluation research may create two basic problems. There are might be disagreements in whether publishing the results and about publication credits.

A final important issue is the availability of funds for evaluative research. When the budget is curtailed, requests for evaluation funds may be among the first to suffer.

Since the ultimate purpose of evaluation is to contribute to the effectiveness of action programs, implementation of research results is a critical phase in the process.

Some of the nonuse of evaluation results is attributed to limitation of the research itself.

Evaluator’s lack of authority. Since the evaluator is an advisor, policy makers are under no obligation to accept his/her recommendations.

Administrators might not understand or appreciate the relevance of evaluation results

Discrepancy between findings of evaluation research and formal evaluation.

Judgments of the competence of evaluators

Disagreements about evaluative criteria. Administrators sometimes claim that the real goals were not measured.

A basic administrative issue concerns the comparative advantages and disadvantages of inside and outside evaluators. The inside evaluator is a staff member in the organization whose programs are evaluated; the outside evaluator is an outside consultant.

Advantages of outsiders are

1. they tend to be better able to maintain objectivity;
2. they may be able to mediate more effectively where thee is extensive internal conflict;
3. they usually are better protected from problems of marginality and status incongruity;
4. they are better able to avoid unwelcome nonresearch tasks;
5. funding agencies, and general public, usually consider the report of external evaluators more credible

Advantages of insiders:

1. they are usually able to develop a more detailed knowledge of the organization and its programs
2. they are in a better position to do continuing research

When evaluation is conducted for the purpose of accounting to an outside body – use outsiders. When it is conducted to assist an organization in its program development effort – insiders.

If the evaluative researcher hopes to contribute to internal program development, he should take early steps to establish effective with those who make key decisions regarding programming.

The evaluator’s prestige and power are considered to be positively related to the likelihood that his findings will be implemented. If the evaluator is an insider, it is important that it has a prestigious position within organization. Similarly, if it is an outsider, it is helpful if it has strong professional and organizational credentials. It is also important for an outsider evaluator to be linked to someone of high status in the action organization – the relationship that is called Merlin Role.

As he begins working with agency representatives, it is important for the evaluator to create what scientists call Image of potential. The evaluator must, for example, provide administrators and practitioners with assurance of integrity his technical competence, his understanding of the action setting, and his personal integrity and decency.

Administrators should be informed about limitations of the contribution of evaluative research. An early agreement regarding the manner in which evaluation results will be published is also desirable and also who gets credits.