Written by: Christian Roth (CSIRO)

During the last week of January 2018, the SIAGI Project Leader, Christian Roth (accompanied by Stefanie, his wife) spent time visiting the SIAGI case study villages Dhaloguri and Uttar Chakowakheti, together with CDHI, the joint SIAGI and DSI4MTF partner NGO.

A primary purpose of my immersion in the case study villages was to experience and test firsthand what some of the challenges are to collecting, documenting and interpreting qualitative data as an alternative to resource intensive and extractive methods such as quantitative surveys. In fact, I regard this exercise as an example for how (qualitative) methods that heed our principles of ethical community engagement can still yield valid and scientifically defensible insights and conclusions. I offer the following observations and principles:

Open-ended questions

It takes discipline not to ask (suggestive) questions seeking to provide the answers to a predetermined view or hypothesis. I found myself on occasion slipping back into asking questions of confirmation to affirm what I already thought. For many of us scientists, articulating open-ended questions is likely a skill that requires active practicing over many interviews or conversations. As long as we are sufficiently self-aware when we slip back into old questioning patterns, this can be managed. Shadowing or being coached by people who are good at this is perhaps one way of getting better at this skill.

Reflective and judgement-free listening

Learning to leave behind what we think as being right or wrong, good or bad, and just taking the information we hear for what it is I think is perhaps the most challenging skill required, and perhaps a skill many of us are not well versed in. We all have our biases, conscious and unconscious. I wonder how we can train ourselves in getting better at this; again perhaps observing others who are good at listening?

Consciously looking

Some people are more visual than others. I personally ‘see’ more than I ‘hear’. As a soil scientist and agricultural scientist, as I do the village walk I look at the state of the soils, the nature of the irrigation being performed, the state of the crops, what crops are being grown in the homestead gardens, the quality of the ponds, the way the animals are kept and what they graze, whether the dung goes into fuel or into manure for the fields etc. Also, as I walk through the village I try to be aware of how people interact amongst themselves, what they wear, whether they look healthy and well-nourished. I look for indicators of wealth or surplus cash – how many TV satellite dishes are on the roofs, motorcycles, smart phones, solar panels, are the houses thatched, brick or tin etc. All of these visual stimuli create a picture of the state of the village and the being of its inhabitants that provides context and subsequent entry points for conversations and questions.

Creating space for the unexpected

If we pre-structure our engagement, we are likely to be more efficient (for example the day spent in UC systematically looking at all four collective sites). But we risk missing or closing out on things we would otherwise see or think about. The contrast on day two in Dhaloguri was revealing – by doing the random walk we ended up having discussions we wouldn’t have had otherwise (eg talking to non-collective farmers and women), at the expense of not having looked systematically at all collectives and their sites. The challenge is finding the right balance between both modes – they complement each other.

Taking time

Perhaps the most striking thing I learnt is that all of the above takes much more time than I thought. Active or reflective listening changes the pace of conversations. They meander. But that is how you create space for the unexpected. The combination of pre-structured discussion and visiting of sites complements the random conversations. Ideally I would have spent two days in each village, doing both.

The other dimension of time that I had underestimated is the value of immediately reflecting on and mulling over what had been said and seen with the CDHI team and Stefanie, over lunch or dinner, in the vehicle etc. The benefit of ‘fresh’ processing of the information, its triangulation, and continuous contextualisation I think cannot be underestimated, and I think this created a most stimulating environment for learning and insights. To some extent this hedges against not having captured all of the notes on the go (see below).

Capturing and documenting the conversations and observations

I tried two methods. On Day 1 in UC, I focussed exclusively on the interactions, leaving the compilation of notes to the evening. This was challenging. After a long day, I was tired, and really had to force myself to write up notes. I probably forgot some of the things I had seen or heard. In contrast, on Day 2 in Dhaloguri I took notes on the go, during the discussions. In this case I noticed that I wasn’t as good in the active listening, hastening my note writing to keep catching up with what was being said. I missed some of the subtleties of body language (leaving aside the problem of not speaking the language and having to rely on translations).

On reflection, I think the best approach is to have two people involved in the discussion or interview – a speaker, and a note taker. In the evening, or back in the office, these two can then triangulate, compare and gap fill. I know that this is standard practice for some of our NGO partners, but I am not sure how the PhD students might be able to do this.

Longitudinal studies of NGO practices

An unexpected learning was the value of extending the project timeline backwards by looking at past intervention sites, to be able to extrapolate forwards as to how certain approaches and interventions might play out. I think this is something we need to think a bit more about in terms of how we capture evidence of long term sustainability of project interventions. I am convinced that all of our NGO partners would be able to take us ‘back in time’ to villages where they have worked in the past. However, for this kind of longitudinal approach to be valid, there would have to be some confidence in the consistency between engagement methods in the past and currently.