By Gordon Jackson - November 2019

There are many proficiency testing schemes available to the food and drink industry. Participation in such schemes provides confidence that laboratory analytical results are meaningful and accurate, helping to ensure consistency in the quality of the food or drinks produced and their safety. There are several suggested benefits associated with proficiency testing schemes, but how do they really affect or benefit brewers? We asked a group of leading technical experts for their feedback on one long-established proficiency testing scheme.

The Brewing Analytes Proficiency Scheme – what it aims to do

The Brewing Analytes Proficiency Scheme (BAPS) is an international scheme designed to promote quality in the measurement of a range of chemical, microbial and sensory analytes in beer, and to help participants monitor and improve the quality of their measurements. BAPS is accredited by the United Kingdom Accreditation Services (UKAS) and is a partnership between LGC Standards and Campden BRI, and has over 350 participants.

The scheme was set up in 1994 with the aim of helping brewers to:

• demonstrate the effectiveness of their own quality systems
• compare their laboratory's measurements with those of their peers
• demonstrate competence to third parties such as accreditation bodies, regulators and customers
• monitor trends in measurements over time
• monitor an individual's capability as part of their training programme

Under BAPS, participants are sent samples to analyse using their in-house methods and the results are sent back to the scheme organisers. Each participant then receives a report comparing their performance with that of all other participants.

Participants can receive samples for the analysis of:

• basic chemical analyses on lager/ale
• more advanced analyses on lager
• basic analyses on dark/craft beers
• basic analyses on alcohol free/low alcohol beer
• analysis of gluten in low-level gluten beer
• microbiological identification and enumeration of common brewery microorganisms
• sensory analysis of lager/ale

Full details of tests undertaken with BAPS are shown in Appendix 1.

So, what do the brewers really use it for?

There are many uses of the data from proficiency testing. On the 25th anniversary of the BAPS scheme we took a look at how its participants use the results (data) on the chemical analysis, microbial analysis and the sensory analysis of beer. The aim was to understand how brewers use the proficiency scheme and the benefits they realise from it. The survey targeted the 11 members of the BAPS Advisory Group, which comprises leading figures from the world’s brewing companies, who steer the scheme to ensure that it meets the requirements of the 350 companies that take part in its proficiency testing.

The key results from the survey were:

For chemical analysis of beer

Samples of commercial beers are sent to participants. Homogeneity is checked before samples are sent to them. As shown in Appendix 1, chemical analysis includes samples for lager/ale, lager, dark/craft ale, alcohol free/low alcohol beers and low-level gluten beers. However, in this survey we did not distinguish between the different samples types and just considered them all under the general heading of chemical analyses.

Here’s how the chemical analysis findings were used:

• Comparison: Most companies used the results to identify tests where good performance is not achieved compared to other participants in the scheme. This enabled them to take actions to improve these analyses
• Training: 73% used the results to identify the need for retraining of analysts and/or to evaluate the effectiveness of the training provided
• Modifications: When companies had amended methods then subsequent performance in the PT scheme was used to check the analyses after modifications had been made (64%)
• Subcontracting: Some of the participants in this survey are involved in subcontract arrangements, so in these cases, BAPS is useful in ensuring that the laboratories with these arrangements produce similar results
• Performance indicators: 27% of participants also set key performance indicators (KPIs) based on the percentage of satisfactory scores in BAPS

For microbiological analysis of beer

The BAPS microbiological test materials contain organisms typically found in the brewing industry. These are sent out as freeze-dried organisms and participants can identify these to a basic level (e.g. bacteria or yeast), intermediate level (e.g. lactic acid bacteria) or advanced level (e.g. lactobacillus brevis). The survey showed that:

• Levels: All participants used the scheme to test the ability of the laboratory to identify and quantify unknown organisms to basic level, 70% identified to intermediate level and 50% to advanced level
• Training: 60% used the results to identify training needs and/or to evaluate the effectiveness of training given to analysts
• Test systems: 30% of participants also used the results in a wider sense to test the overall systems for the aseptic handling of microorganisms in their laboratories
• Subcontracting: As with chemical analyses, the results were used by subcontract brewers and packers to prove to their clients that their microbiological data was reliable and in line with results from other breweries
• Performance indicators: 20% of participants set KPIs based on correct identification/enumeration of microorganisms in BAPS

For sensory analysis of beer

Participants in BAPS receive samples of commercial beers; they evaluate various aroma and taste characteristics and compare their results (both quantitatively and qualitatively) with the consensus of other taste panels and also with the performance of individual tasters in other taste panels. The results of the survey showed that:

• Quantitative comparison: For quantitative scoring of the intensity of the aroma and flavour, 50% used it to compare the consensus of their taste panel with others in the scheme while 66% used it to monitor the performance of each individual taster
• Qualitative comparison: For qualitative comparison (i.e. testing the ability of the taste panel overall and of the individual tasters to detect key aroma and taste characteristics) 83% monitored the overall performance of their taste panel while 66% monitored the performance of individual tasters to identify aromas and flavours
• Subcontracting: Once again, the results of sensory proficiency testing were used by subcontract brewers and packers to prove that their tasters and taste panels could correctly identify various aromas and flavours in a range of beer types and that their results were aligned with the taste panel of the client and of other taste panels

Other uses

In addition to the specific uses mentioned above, some further and generic uses were identified by participants, including:

• Confidence: most participants used the results of proficiency testing to provide confidence that analytical, microbiological or sensory data is meaningful and accurate
• Compliance: 70% used the results to demonstrate compliance to standards such as the BRC Global Standard for Food Safety
• Quality management systems: 60% used the results to assess the effectiveness of their overall quality management systems
• Accreditation: 40% also made use of BAPS being accredited to ISO / IEC 17043:2010

Conclusion

The results of the survey show the variety of uses for results from proficiency testing schemes:

• comparing performance with others in BAPS - with 350 breweries taking part this gives a very good comparison and sound statistical data
• using BAPS for training staff also figured highly. This could be useful for checking the performance of newly trained staff.
• for breweries that are involved in subcontracting of brewing/packing operations BAPS is a good way of ensuring that the results from the labs on both sites are aligned as well as with the rest of the brewing industry
• taking part in PT schemes can also be helpful with compliance to standards such as the BRC Global Standard Food Safety. Clause 5.6.2.4 of the standard requires procedures to be in place to ensure reliability of laboratory results; including the use of a system to verify the accuracy of results (e.g. by ring trials or proficiency testing). For breweries, BAPS provides an ideal means of achieving this.

Appendix 1

The full range of tests available with the Brewing Analytes Proficiency Scheme (BAPS):