THE OUTLINE

Background

1) Veterinary laboratories are regarded by many as the backbone of veterinary services and play a crucial role in the risk assessment. In most countries, veterinary services function as a risk management system, which is an important component of the requirements for international trade compliance. However, the role of veterinary laboratories cannot be fulfilled unless an adequate quality assurance programme (QAP) has been established.

The overall objective of a QAP is to provide confidence in the validity of the laboratory’s test results and services.

2) A laboratory QAP begins with the formulation of a set of quality goals for each test result. Once the quality goals are defined, the QAP works by repeatedly comparing the laboratory test results with the predetermined quality goals in order to identify and correct any deviation from the quality standard, usually documented in the quality manual.

The ability of a laboratory to identify deviation from quality standard or detect changes in the quality of results is based on dedicated internal and external monitoring procedures.

3) Inter-laboratory test comparisons (ILC) are a form of monitoring procedures from outside the laboratory. Being an external quality assurance procedure (eQAP), ILC provides among others unbiased external measure of the accuracy of laboratory procedures. For instance, when performing procedures such as serology, whose test results are measured on continuous interval scales; an eQAP can detect slowly developing systematic errors that may be missed with internal quality control procedures.  For laboratory procedures providing discrete categorical results, such as bacteriology, eQAP is sometime the only objective check on the accuracy of the results.

4) Laboratories may undertake ILC for various reasons, which could be one or more of the following:

 

i) Determining a laboratory’s capability to conduct specific diagnostic tests

 

ii) Checking or certifying the performance of individual operators. Many laboratories are using the results of ILC as the basis for authorizing heir personnel to perform certain tests and operate certain equipment items.

 

iii) Checking or certifying the calibration of instrumentation. The calibration status of laboratory equipment should always be verified and maintained according to a predetermined schedule.

 

iv) Harmonizing existing test methods. For instance, the SADC Diagnostic Sub-committee of the Livestock Technical Committee has recently used ILC to evaluate the suitability of the harmonized standard operating procedures (SOP) from 14 member states, including avian influenza HA/HI, conventional and real time PCR and Rabies FAT SOPs, etc.

 

v) Evaluating new test methods

 

vi) Assigning values and ranges to standard materials

 

vii) Resolving inter-laboratory differences

 

 

Proficiency Testing

1) Introduction

 1. Proficiency testing (PT) determines a laboratory's capability to conduct specific diagnostic tests. As an ILC, PT is conducted for the express purpose of determining a laboratory’s capability to conduct specific tests and it must involve at least two laboratories.

 

Proficiency testing is an integral part of laboratory accreditation programmes. An accreditation programme is a formal process for recognition of laboratory quality and capability by an independent authority (e.g. SANAS in South Africa and SADCAS in the SADC region), which awards or denies accreditation based on stipulated requirements for quality and capability.

 

In South Africa, the veterinary authorities have made PT compulsory for tests for controlled diseases, as part of the criteria used to determine laboratory’s capability/approval.

 

2. Benefits for participating in PT are enormous. Proficiency testing provides benchmarks and gives participating laboratories an opportunity to determine their individual performance as compared to other laboratories performing similar tests.

 

Inter-laboratory test comparison can be used to determine the performance of a method (ISO/IEC 17025:2005), as an extension of the validation criteria (OIE Terrestrial Manual) or a means by which the validity of the tests undertaken is monitored.

 

3. JGK Lab AfriQA Proficiency Testing Scheme (PTS) JGK Lab AfriQA PTS makes use of well-characterized test materials/check samples, which are simultaneously sent to participating laboratories for testing.

 

The results are collected and analyzed against the intended result in order to determine the capability of a participating laboratory to conduct a diagnostic test and produce correct results.

 

 

2) Scope

The ultimate goal of JGK Lab AfriQA PTS is to cover all diseases and conditions of veterinary importance occurring in Africa, but activities will be discharged in phases.

 

 

3) Design

1. JGK Lab AfriQA PTS provides each participating laboratory with check samples made of aliquots or replicates of the same specimens.

Participating laboratories analyze the samples for one or more variables, as defined.

 

The results of the participating laboratories are returned for data analysis to JGK Lab AfriQA. Reports summarizing the results are distributed to

the participating laboratories 3-4 weeks after submission. The reports consist of statistical summaries and plots indicating how the results of the

participating laboratories compare with the results of the peer group. In most cases the true value, or benchmark, are used as the mean value of the group for each analysis. In others, the benchmark is the value obtained by a standard reference method.

 

2. Proficiency testing scheme, like accreditation programmes, must be operated by an independent and competent authority in order to prevent any bias and to generate reliable data. JGK Lab AfriQA undertakes to comply with the requirements of ISO 17043 and OIE guidelines, as minimum standards, in carrying out its PTS activities, and to achieve SANAS accreditation within a reasonable time-frame. JGK Lab AfriQA also undertakes to abide by its code of conduct not to disclose to a third part any PT results without the consent of the participating laboratory or country authority.

 

 

4) Organization and Management

1. A dedicated official within the ranks of JGK Lab AfriQA is assigned the duties to oversee the entire operation, quality and security of the PTS. This official is also responsible for ensuring that check samples that are not produced by JGK Lab AfriQA itself are manufactured, through subcontracting, by laboratories that fully comply with the relevant requirements of the ISO 9000 series of standards and ISO/IEC 17025.

 

2. All personnel involved in the PT shall operate free from pressure or inducements that might unduly influence the analysis of PT results. They must provide adequate supervision and security in the production and distribution of check samples to be used in the PTS, and receipt and analysis of test results submitted by participating laboratories.

 

3. For the characterization of test materials to be used as PT check samples, JGK Lab AfriQA uses OIE prescribed methods or other standard methods of identical diagnostic performance characteristics.

 

 

5) Selection of PT check samples

1. The selection of test materials for inclusion in the PT check samples is done in two phases:

 

(i) JGK Lab AfriQA or an accredited subcontractor conducts the initial assessment of test materials by using standard methods to determine the status and/or reactivity of the sample;

 

(ii) the samples are repeatedly tested by at least two analysts conducting multiple runs on different days.

 

2. No sample is included in the PT check samples before sufficient values are generated to assure the unequivocal status of the test material, including homogeneity.

 

 

6) Composition of the PT check samples

1. The number of samples making up a panel for distribution to participating laboratories is dictated by the type of analysis to be performed on the results and the numbers required to ensure statistical validity. JGK Lab AfriQA’s statistician provides the necessary guidance needed in this regard. For antibody detection, irrespective of the type of test, the minimum number of samples to be included in a panel is three.

 

These three samples are made of:

 

(i) one unequivocal strong positive,

 

(ii) one unequivocal weak positive, and

 

(iii) one unequivocal negative.

 

2. More samples which vary from one PT round to the next are added to the abovementioned panel to prevent participating laboratories from anticipating the expected outcome when receiving three samples of this nature after a few rounds of PT. The additional samples may be completely different from the three; they could also be their replicates or a combination.

 

 

Statistical Analysis

1) Types of data

JGK Lab AfriQA’s PTS is designed to generate both qualitative and quantitative data which are subjected to further analysis. Qualitative data are data such as 'positive', ‘present’, 'negative', ‘absent’ and/or 'suspicious'. No statistical procedures are applied to these data. Quantitative data (e.g. end-point titres) are analyzed using a range of available statistical tools.

Irrespective of the type, the data submitted by participating laboratories must be compatible in order to be analyzed. Therefore, clear instruction to be followed by the participating laboratories when analyzing the samples will be given by JGK Lab AfriQA’s PT team.

2) Assigning values to the check sample panel

1. The preliminary value, range or status of the test material is assigned by JGK Lab AfriQA or its subcontractor during the production and selection phase.

The status of samples generating qualitative data is verified intensively to ensure that it is in fact correct since it may be the only acceptable value. When the results of at least 80% of the participating laboratories agree with the assigned value, it (the assigned value) becomes irrevocable. Any other value obtained which is different from the assigned value is classified as wrong and has nothing to do with the homogeneity or stability of the sample.

For quantitative and semi-quantitative data, the value assigned by JGK Lab AfriQA is recalculated after PT results are submitted, and it is taken as the mean value after removal of outliers.

2. JGK Lab AfriQA may also assign the target value on the consensus value of the returned participants’ results.

3) Statistical methods

JGK Lab AfriQA uses valid and straightforward statistics that are meaningful to the participating laboratories. They include but are not limited to:

(i) Frequency analysis, for participating laboratories to see where their performance lies with respect to the other laboratories in the PTS,

(ii) Measures of intra- and inter-laboratory variance through repeatability and reproducibility in-dices to provide valuable information on the precision and robustness of the test methods,

(iii) Youden analysis to indicate systematic or random error sources that may be causing problems in individual laboratories, etc.

Pass / Fail Criteria

1) Ideally, the setting of pass/fail criteria for tests for infectious diseases should be directed by country authorities depending on diseases dealt with and how crucial it is that a positive animal/case should never be missed or a healthy animal should never be misclassified as infected. It will take into consideration factors which may vary from one disease to another and between types of tests.

 

Any result of tests signaling a problem with diagnostic sensitivity or diagnostic specificity should be carefully considered.

 

 

2) JGK Lab AfriQA will get inputs from statisticians and epidemiologists in deciding on the types of statistical analyses to be used in making pass/fail decisions.

 

Laboratories submitting results that fall outside ranges established by statistical means will be identified.

 

 

Frequency of Proficiency Testing

1) JGK Lab AfriQA’s PTS will run throughout the year. However, the frequency will depend on diseases / conditions and requires inputs from stakeholders and country authorities.

 

 

2) Laboratories showing some problems in certain areas are encouraged to participate more frequently in order to assess the effectiveness of the corrective actions undertaken.

 

 

Laboratory Recognition

1) The recognition of a laboratory’s performance through PTS also depends on diseases dealt with and requires inputs from regulatory authorities. For instance, DAFF is the only authority in South Africa which can authorize or deny a laboratory to perform tests for infectious disease(s) based on a set of predefined performance criteria.

2) If required, JGK Lab AfriQA, in consultation with stakeholders and country authorities, will define and document criteria for awarding, denying or withdrawing recognition.

Logistics

1) Eligibility and Acceptance

Testing laboratories conducting tests for the diagnosis of infectious diseases and microbiological investigations are all eligible. Beside this outline, JGK Lab AfriQA will send out further information pertaining to frequency and dates of testing, commitments and deadlines, methods of data analysis, reporting structure, disposition of results and confidentiality.

In addition, a form to be signed and returned to JGK Lab AfriQA will be included which indicates that the laboratory accepts the terms and conditions of the programme.

2) Notification and Shipment of Panels

1. JGK Lab AfriQA will notify participating laboratories at least one month in advance of pending PTS. The notification will include the projected date and method of shipment of the sample panel. Participating laboratories outside South Africa requiring import permits for the check samples should notify JGK Lab AfriQA so that longer notification is provided.

2. Check samples will be uniquely coded. Their status or expected result will never be known to participating laboratories.

3. All shipments will comply with IATA regulations concerning the shipment of biological materials.

4. Upon shipment, JGK Lab AfriQA will inform the recipient laboratories telephonic-ally or by email of pertinent details in order to facilitate rapid retrieval and clearance of the shipment upon arrival.

5. Check samples arriving in a damaged or questionable condition will be replaced immediately.

3) Testing and Return of Results

1. JGK Lab AfriQA will give participating laboratories enough test material and enough time to complete the testing of the PT sample panel to their satisfaction. Participating laboratories are free to test the PT check samples more than once and by more than one analyst.

However, only one set of results (not the mean of results of various analysts) will be accepted by JGK Lab AfriQA for analysis.

Participating laboratories are encouraged to let the analyst responsible for running the test routinely to run the PT check sample to reflect the true picture of the laboratory capability.

2. The PT check samples will be accompanied by a complete set of instructions with respect to reconstitution, storage and handling, special testing requirements, data expression and deadline for the submission of results.

3. Results must be returned in the proper format and on time.

Failure to do so could lead to omission from the round of PT and loss or downgrading of recognition status if the system allows it.

4. JGK Lab AfriQA will acknowledge receipt of the results and their acceptance into the analysis.

4) Analysis and Reporting

1. Timeframe for completion of analysis and reporting is 3 to 4 weeks after the deadline for the receipt of results. A general report summarizing the results of all of the analyses will be prepared for distribution to all participating laboratories.

Participating laboratories will be randomly assigned a code to ensure anonymity in the general report. Individual laboratories will be informed of their unique code for each run of PT and, where applicable, also receive a summary of their own performance.

The performance will be compared from one run to another. Where there is an indication that the performance of the participating laboratory has dropped, JGK Lab AfriQA may assist the participating Laboratory to identify the causes, after which, and with their consent, another panel of PT check samples may be re-issued to check the effectiveness of the corrective actions taken.

2. The reports will come with a statement of performance and an official certificate of participation in the PTS will be issued to the participating Laboratory at the end of the cycle.

3. All data, results of analyses and the performance status of participating laboratories will be kept in confidence at all times. JGK Lab AfriQA will not divulge this information at any time, unless at the request of or with the consent of the participating laboratory or country/relevant authority.

1) James EC Bellamy & Dennis W Olexson:

       Quality Assurance Handbook for Veterinary Laboratories, Iowa State University Press, 2000.

2) OIE Quality Standard and Guidelines for Veterinary Laboratories:

       Infectious Diseases, second edition, 2008

3) ISO/IEC 17043:2010:

      Conformity assessment – General requirements for proficiency testing