2. EVALUATION METHODOLOGY

• 2.1 Spatial Domain
• 2.2 Temporal Domain
• 2.3 Sample Size
• 2.4 Data Collection
• 2.5 Data Entry, Processing and Analysis

[page-number of print-ed.: 9]

2. EVALUATION METHODOLOGY

2.1 Spatial Domain

To assess the extent and impact of the inputs on crop productivity, and the effects of the programme on technological adoption among the small holder farmers it was felt important that evaluation data be collected in all the rural provinces covering all the agro-ecological regions of the country. This physical diversity enabled the evaluation to look at all the crops and inputs involved in the crop pack programme and facilitated comparative observation of the different crops supplied by government.

Twenty (20) districts were selected, through purposive sampling, from the eight provinces. This technique was applied in order to cover all the agro-ecological regions. A uniform number of thirty (30) farmers were to be interviewed in each of the twenty districts irrespective of their sizes. This was also done to give us a proportionate numerical representation of the agro-ecological regions identified and selected. These farmers were to be chosen from all the three smallholder agricultural subsectors, namely: communal, small scale commercial and resettlement areas. The major reason for this spatial focus is most obvious: the capture of most of the subsectoral variations, peculiarities and problems.

2.2 Temporal Domain

The crop pack programme has now been running for the past five seasons, i.e. from 1992/93 to 1996/97. It was felt that an effective evaluation of the impact of this programme could not be achieved by assessing the performance of only one or two seasons. Consequently, the temporal focus covering the whole programme period was chosen, and hence salient data were collected for the entire period of the programme.

It was, however, accepted that some of the historical statistics, connected with the first two or three seasons, could not easily be retrieved from some small holder farmers who might not be used to record keeping and/or may be illiterate.

[page-number of print-ed.: 10]

The timing of information gathering was initially planned to cover the period immediately after the planting season when farmers' memories on the 1996/97 crop pack programme distribution were expected to be still vivid. But the planning procedures and preparations of the evaluation protracted to the end of February 1997, so that data collection was eventually done in March 1997. However, as it turned out, this delay was providential because the evaluation was able to capture all the information on the timing of inputs distribution for the current season, some of which was not distributed until February 1997.

2.3 Sample Size

The potential target population of the crop pack programme beneficiaries is estimated to be some 1.2 million households. Although the actual target farming population, according to government selection criteria, can substantially be smaller than 1.2 million, available resources - i.e time, money and manpower - could not permit the contact of the whole target population. A numerically manageable spatial domain, covering only twenty (20) districts with a total sample size of some 600 smallholder farmers, was chosen. Table 2.1 shows the distribution of the evaluation sample by province, by district, by subsector, and by agro-ecological region.

Six hundred (600) farmers constitute about 0.05 percent of the total smallholder farming community. By general research standards this percentage is rather small. But the similarities and homogeneity of the operations of, and the problems besetting smallholder agriculture made the evaluation team felt the sample fairly adequate to allow the emergence of general trends and patterns from which some generalisations and conclusions can be inferred with regards to the programme.

Simple random sampling technique was used to select most of the 600 farmers to be interviewed.This technique was used to allow each individual member of the selected districts' farming population an equal (non-zero) chance of being included in the final sample. This made the sample fairly representative of the parent population, thus permitting the making of reasonably valid and reliable generalisations about the findings of the evaluation

[page-number of print-ed.: 11]

with a measurable amount of confidence. The majority of the wards chosen have up-to-date lists of households with both the Councillor and Agritex personnel. These lists offered reliable and accurate sampling frame for the purpose of the evaluation random sampling.

In a few districts, particularly those which combined two farming subsectors (such as communal and small scale, or communal and resettlement), two or three sampling techniques, such as stratified and random and/or systematic sampling, were applied in the selection of the evaluation sample. In addition to information from the 600 farmers, valuable data were also collected from 20 distribution committees responsible for the wards which were chosen in the 20 districts. These 20 committees were selected on the basis of the clustering sampling method used in choosing the wards which were identified with the different agro-ecological regions.

The evaluation team felt it important to collect specific background information on the philosophy, history, components and objectives of the crop pack programme. Specific data on the magnitude of capital injection into the progamme are also necessary. So the Ministries of Agriculture and of Finance, and the Department of Agricultural, Technical and Extension Services, which are at the centre of the implementation of the crop pack programme were selected to be interviewed.

2.4 Data Collection

The major data collection method used to gather information which forms the basis of this evaluation report was the 'questionnaire'.Three different types of questionnaires were designed and used to gather primary data on the background of the programme, on its administration and implementation, on its impact on crop productivity and technological adoption, and on the major problems experienced in the targeting, management and distribution of the inputs.

[page-number of print-ed.: 12]

TABLE 2.1: Distribution of Evaluation Sample (N=600)

SOURCE: Evaluation Plan, 1997

The first questionnaire (Appendix II) was administered to 602 smallholder farmers, distributed as shown on Table 2.2 below, with nearly two- thirds of them (62.3%) coming from the communal sector.

[page-number of print-ed.: 13]

Table 2.2. Distribution of Responses By Sector (N=602)

SOURCE: Evaluation Results, 1997

The evaluation team understood that a considerable proportion of the target population may not be able to read nor write, and that the majority of those who are literate might, for a variety of reasons, not accurately fill the questionnaire and return it in time. Moreover, postal services in some rural areas are extremely unreliable and in some parts of the country are almost non-existent.

The use of mail questionnaires for gathering data was, therefore, out of the question. Consequently, a special team of 20 enumerators from the Zimbabwe Farmers' Union field staff was constituted to interview each of the planned 600 farmers in the sample, and recorded the responses in the questionnaire. This personal interview approach reduced the need for literacy among the sample, overcame possible farmer reluctance and potential resistance to answer certain questions, and yielded much more satisfactory results than a mail questionnaire from an unknown source could have produced, especially among the Zimbabwean peasantry where interaction is so highly personalised. This approach also enabled supplementary questions to be asked to obtain additional information, to explain certain answers and to check that the respondents were not making up their responses.

The second questionnaire (Appendix III) was administered to distribution committee members representing all the 20 districts

[page-number of print-ed.: 14

which were selected. Table 2.3 shows that most of the distribution committees consisted of the local Agritex staff, local councillor(s), village heads, Zimbabwe Farmers Union leadership, Zimbabwe African National Union (Patriotic Front) local leadership and specially elected office bearers for the input distribution purposes. A few distribution committees also include some district leadership categories, such as the police, the District Administrator's Office, the District Development Fund Office and teachers, among their committee membership. This second questionnaire covered some questions that dealt with largely similar aspects as those dealt with by the first questionnaire. Such data from the second questionnaire for distribution committees were important for comparative purposes with those provided by the farmers. This helped to check the accuracy of certain statistics and claims given by the farmers as some of them had no written records of the data of their past inputs and outputs.

The third questionnaire (Appendix IV) was administered to government officials of the Ministries of Agriculture, and of Finance, and the Department of Agricultural, Technical and Extension Services. The two Ministries are important to the programme in their role for policies and financing respectively. Agritex is central in its responsibility for the field administration and implementation of the whole programme. The questions in this third questionnaire were pertinent to the capture of background data on the crop pack programme as alluded to above. The information acquired by the third questionnaire provided an interesting insight, in the introduction to this report, into government policy and commitment to the development of smallholder farming sector.

2.5 Data Entry, Processing and Analysis

It has already been stated that the evaluation assisgnment used three different types of questionnaires. The first one, administered to the selected 602 farmers, was generally structured with closed and precoded questions. In the main, the questions in this questionnaire sought quantitative agricultural data. The second questionnaire, administered to 20 distribution committees, had mainly open ended questions, with no preset responses.

[page-number of print-ed.: 15]

The justification for having closed questions in questionaire 1 was to elicit standardised responses which could be quantified. In the questionnaire for distribution committees the evaluation team preferred the open ended type of questions to facilitate the capture of a full range of responses from the implementators of the programme.

Table2:3 Composition of Distribution Committees (N=20)

SOURCE: Evaluation Results, 1997

Although the responses from the open-ended questionnaire were not standard, the evaluation team was comfortable with the rich quality of the collected data.

The third questionnaire, administered to government, had a mixture of open and close-ended questions.The closed questions

[page-number of print-ed.: 16]

were designed to gather quantitative data on the input and

production statistics which were considered vital for the evaluation The completed data for questionnaire 1 were entered into the computer. Since the responses were already coded, the process was easy and fast. It was not so with the second and third questionnaires where the evaluation team had to first go through the responses to determine patterns and trends. Once these were determined, the next step was to device codes, a most tedious and involved procedure. Much time was lost in classfying responses. Once their classifications were done, then followed data entry, processing and analysis, using a statistical package, SPSS.

The processing of open-ended questions led to some delays because much time was spent on trying to make sense of, and devicing codes for the responses. But it is a process which it was resolved to accomplish in order to get reliable data from which meaningful interpretations could be inferred.