Evaluating Bible Translation

What are the indicators and processes for evaluating Bible translation and who is responsible for carrying them out?

The current Bible translation landscape brings together a variety of stakeholders, including passionate individuals, churches, community organisations and international agencies, united by a desire to see communities and individuals transformed by the Word of God. Some devote their lives to this at great personal risk, while others bring their technical expertise or financial offerings. How can these stakeholders know that they are investing wisely, and that their time, expertise and money are leading to the impact they desire? What are the indicators and processes for evaluating Bible translation and who is responsible for carrying them out?

Over the past century, many Bible translation agencies have focused on the development of ‘translation consultants’, technical experts in language, biblical studies and translation, who play a crucial  role in the quality assurance of Bible translations by training others and checking translated material. Such consultants have very often been the primary way that external stakeholders have verified that Bible translation is being done well. For example, satisfactory consultant reports have typically been a requirement before a publisher will agree to publish a written translation, with these reports often relying heavily on checking done towards the end of the translation process. 

These consultants work in different ways within different organisations and across different contexts, so it is important not to assume all consultants work the same way, nor that they only provide end-stage quality checking. Some provide quality assurance support to a team throughout a project, others are only contracted for a specific time period. Some provide dedicated support to a single team, others foster group consultations, while others run workshops or courses in training institutions. Some rely heavily on back translations and software tools, others emphasise face-to-face connections with the intended audience to more directly evaluate comprehension and impact. Some work cross-culturally, perhaps offering remote support for areas that are difficult to access, others are embedded within their own sociocultural context. 

In November 2023, the ETEN Innovation Lab offered three recommendations for quality assurance in Bible Translation¹, related to multimodality, the church, and iterative processes. Some of these recommendations recognise best practices that have been developed over many years, others would be less familiar in some contexts.  This paper looks at the impact of these recommendations on the evaluation of Bible translations, and has particular relevance for those organisations that have invested in developing consultants, and for contexts where a high priority is placed on quality checking at the end of the translation. What might these recommendations suggest about how consultants engage with Bible translation in the current landscape?   

1. Multimodality

The first recommendation is for multimodal translation processes. This is built on the recognition that communication is fundamentally multimodal, utilising speech, gestures, writing, music and images.  Our various senses, our embodied ‘whole-being-ness’ and our lived experiences all matter in the way we make meaning from any interaction. If translation is built on multimodal processes, such as embodied methods to familiarise yourself with a text, internalise it and then perform it in your own language, quality evaluation frameworks also need to be multimodal. For comparison, an evaluation of a multitrack performance would be very limited if it were only based on a mono output. Similarly, any evaluation of Bible translation is enriched by a fuller experience of the translation.   

These are a few possible implications of multimodality for evaluation of translation:

• Evaluation needs to take place throughout the whole of the embodied, experiential process of translation, rather than being seen as just a ‘check’ provided at later stages.

• Evaluation criteria need to include the whole range of embodied responses, such as perceptions of aesthetic quality and emotional engagement. They need to be focused on people, not just products. 

• Evaluation requires engagement with the intended modalities. For example, an evaluation of a Sign Language translation is going to be more beneficial if it is based on a performance or video than on a written back-translation. Even written translations are typically intended to be read aloud, so hearing them read strengthens evaluation. 

• Multimodal evaluation may include activities like listening to an on-the-spot oral back translation for a passage even when a written one has been provided.

• An individual’s ability to evaluate a translation is constrained by their level of experiential, embodied knowledge of the languages and cultures that the translation mediates between. This is a motivation to prioritise those who have such experience in evaluation (for example members of the local church and community), and for those from outside the community to recognise how much greater their contribution can be the more they develop such an experiential understanding.

2. Church

The second recommendation affirms that the local church be at the centre of decision-making, resourcing and quality assessment processes of Bible translation programs, and that the local church should know the variety of expertise that is available through the global church and Bible translation agencies. The recommendations expand this by contending that ‘the church is best served when global church experts focus on growing the church’s capacity for and expertise in translation and its ongoing review and revision. When global church experts are invited by the church, their best service is as advisors, trainers, encouragers, and mentors throughout translation’.

What does this mean for external stakeholders seeking to evaluate their investment in translation? If the local church is at the centre of decision-making and quality assessment, they are the ones primarily responsible for deciding whether something has attained the desired level of quality, not external stakeholders alone. This might look different from context to context: in some places this evaluation might be the responsibility of a single church or denomination; elsewhere it might involve collaboration across denominations to endorse Scripture. 

For example:

• In Milne Bay in Papua New Guinea, inter-denominational ‘Scripture Review Boards’ have been convened by the United Church to ensure that quality processes have been used during translation and to endorse (if appropriate) the resulting products. 

• In Madagascar, translation projects originally initiated by the FJKM Presbyterian Church are now administered by a college of leaders from different congregations and denominations. A translation agency was invited to provide consultants to perform final quality control checks during the first phase, whilst training up local consultants to provide intermediate checks and lead final checks in later phases. Evaluation is also provided by church leaders who give exegetical feedback after drafting, and by feedback to readings on the radio straight after the consultant stage.    

• In Nigeria, there are projects where an interdenominational team of pastors and church-leaders conduct the final verse-by verse check, with oversight from a ‘Christian Advisory Council’ that provides final authorization.

• In some projects in Liberia, a college of pastors is taking responsibility for final approval of a draft that was generated and refined by a large team of community members and pastors. 

• For the British Sign Language project, an interdenominational board of trustees takes responsibility for quality assurance processes, with checks carried out by various scholars and BSL experts.

• In a Eurasian context, a single denomination has taken on responsibility for the quality assurance of a translation project, because the security context makes interdenominational efforts quite challenging.

In some contexts, the local church may feel it has sufficient technical expertise amongst its own members to carry out quality control checks to the level it desires. In other contexts, when the church is aware of the expertise available from the global church, they may invite in technical experts to build capacity through training and mentoring, or to work alongside the church in quality control checking, as has happened in Madagascar. 

If external stakeholders are seeking to evaluate their own contributions, this recommendation urges that such evaluation should be based on criteria and processes that make sense to the local church and that they should be fully engaged throughout.

This recommendation leaves some questions hanging. For example, what happens where there is no local church within a language community, or if minority-language speakers are marginalised in decision making by majority-language speaking leaders of the local church?  What happens when different local denominations disagree over the perceived quality of a translation? Or when there are differences of opinion between representatives of the local church and of the global church that have been invited to contribute? Dialogue is key for the way forward. Evaluating sustainable Bible translation requires investment in humble, mutual relationships between individuals, churches and agencies, as well as the development of technical skills and efficient processes.  

3. Iterativity

The third recommendation urges for iterative quality assurance processes and progressive publication. This is based on the perspective that Bible translation is a sustainable, ongoing process rooted in the church. It is an activity that is ever-present and iterative as God’s people in each location seek to contextualise God’s word, allowing God to speak with transforming power into each new generation. With this perspective, Bible translation may be viewed best as a wheel that keeps turning, with ongoing cycles of drafting, use and revision within the church and community, albeit with certain checkpoints in these cycles. This contrasts with a more linear approach to Bible translation, in which particular steps are followed to reach an end goal. 

The linear model can lead to a product-orientation, with evaluation being based almost exclusively on a linear sequence of quality control checks until the product is completed, including team checks, community checks and consultant checks. These checks often focus on qualities of the product, seeking to ensure the highest possible standards in trustworthiness or accuracy when compared to the source, in the use of language that is natural and appealing, understandable to the intended users, and in matching the intended brief for the product. This approach to quality assurance may be compounded by scarcity factors, with projects needing to be completed within certain constraints: limited time (either because of the natural lifespan of key personnel, or goals to achieve targets by a certain date) or limited finances (for example, concern that there is only money now to pay translators, or sufficient funding to warrant just one print run).

In the iterative model, evaluation needs to move beyond checking whether a product has reached the required quality level to be deemed ‘finished’, to include assuring the quality of the ongoing process that is being used and whether there is a suitable abundance of resources (both human and financial) and support to keep the wheel turning into the future. Traditional boundaries between activities that are ‘translation’ or ‘Scripture engagement’ may need to be rethought. Is drafting and revising continuing within the church and community? If not, where are the barriers? Are new translators developing the skills, attitudes and knowledge to keep the translation turning for the next generation? Are appropriate resources available and are they being used effectively? Are translations being used, engaged with, and improved through iterative revisions? 

Several of these points are picked up in the fourth recommendation from the Innovation Lab, which emphasises the existence of ongoing iterative processes, connections to others in a network for support and ongoing training, and active use of Scriptures as indicators of quality, beyond just completing text products.  

Where consultants are involved in Bible translation, then, they need to be equipped not just to speak into the quality of products, but also to support the development of the quality assurance community that surrounds and breathes life into the translation project.  As these consultants seek to serve the church in its iterative, multimodal translation processes, this requires careful thinking about the breadth of services that can be offered (beyond just ‘checking’) and the way these services are offered to local quality assurance communities.

1.  The Lab's fourth recommendation related to "Meeting the All Access Goals in Church Based Bible Translation" and focused less on quality assurance processes.