Since 2005, marketing authorization holders of medicines are required to have the patient information leafets for their products readability tested. The European Directive 2004/27/EC (a revision of Directive 2001/83/EC) defines it as follows: ‘The package leaflet shall reflect the results of consultations with target patient groups to ensure that it is legible, clear and easy to use’ .
Since 2005, marketing authorization holders of medicines are required to have the patient information leafets for their products readability tested. The European Directive 2004/27/EC (a revision of Directive 2001/83/EC) defines it as follows: ‘The package leaflet shall reflect the results of consultations with target patient groups to ensure that it is legible, clear and easy to use’ (art. 59-3) and ‘The results of assessments carried out in cooperation with target patient groups shall also be provided to the competent authorities (art. 61-1). This requirement is for medicines that are newly registered or that have been considerably changed. However, the British authorities (MHRA) require that patient information leaflets for all products that are currently on the market will have to be readability tested before July 2008.
Of leaflets for medicines that are registered for all EU countries through the so-called central procedure (CP), just one of the 21 official EU languages has to be tested. In most cases this will be the English version, but that is not strictly necessary. To reduce cost and perhaps to escape longer wait lists for tests in certain countries, registration holders can have a translated version of the leaflet tested in for example The Netherlands, or in another EU country. Of course the test report will have to be in English so that it is easily accessible for Registration Authorities in other countries and for the European Commission. In so-called mutual recognition procedures (MRP), one of the concerned languages has to be tested, preferably the official language of the Reference Member State (RMS). Often this can be Dutch, as (according to their Annual Report for 2005) the Medicines Evaluation Board in The Netherlands (CBG-MEB) is involved in around 25% of all registration procedures in Europe. The risk of testing just one language (of potentially 21 EU languages) is that the readability of a tested leaflet can be ‘greatly reduced’ by a poor translation. It is therefore of crucial importance that translations are done by professional medical translators. In national procedures, the language of the country may be used, and also the readability test report may be in that language.
There is no absolutely clear information on how readability tests should be performed. The regulatory authorities need to be able to assess that the leaflet was tested in a professional and well-defined way and that persons who might need to use the leaflet would be able to find the requested information, understand it and act appropriately. According to the British MHRA, at least 90% of test participants should be able to find the requested information, and of those again 90% should understand the information. Other EU countries require that at least 16 of 20 testers answer all questions correctly, but in the recently published draft of the revised EC Readability Guideline (which has been under consultation until November 30, 2006, and will be published in 2007) this option is no longer open. As the European Directive uses terms such as ‘readable’, ‘clear’ and ‘understandable’, it is still not totally clear how readable a leaflet has to be.
One thing is clear and that is that leaflets have to be written conform the standards that the Quality Review of Documents (QRD) group of the European Medicines Agency (EMeA) has defined (the QRD template). Using the template, however, can lead to readability problems. Some of the recommended headings and phrases in the template frequently lead to questions during readability tests. Templates in all EU languages can be downloaded from the EMeA website.
What happens during a readability test?
Leaflets are tested for readability by at least two test panels of 10 test participants each. Of each group at least 8 out of 10 participants should be able to locate and understand the requested information. The leaflet should be tested in the format that resembles as much as possible the version that eventually will be packaged with the medicine. If after the first test round the leaflet is revised (which is often the case even if the score is already high enough) the revised version will be tested during the next test round.
Readability of a leaflet is assessed on the basis of the answers the test participants give to a series of 15 questions. If in the first test group at least 8 out of 10 have found and understood the information, this means the readability meets the criteria. The second test round then serves to confirm these results. In case during the first test round less than 8 out of 10 test participants have found and understood the requested information, the text needs to be revised and during the second test round it will be tested if the revisions are making a difference.
The test usually takes between 30 and 45 minutes and is conducted in the format of a person-to-person interview, either in our office, at the patient’s home, or at a different location.
In order for us to find out if the questions can trigger the correct answers and whether perhaps there are any obvious problems in the PIL, we first organise a pilot test with a limited number of participants. Based on the results of this pilot test, we can adapt the questions and make necessary changes to the PIL before the start of the actual test.
Composition of test panels
Test panels should reflect the population for which the medicine is intended. In many cases this means that ‘the average person’ will do. Test panels need to have a balanced mix of men and women, young and old, and with all levels of education. So far we have noticed no difference between men and women, but younger test participants find more correct answers, and quicker, than elderly testers. For specific medicines we need to recruit patients who suffer from a certain disease.
For the test participants it is important to know that it is the leaflet that is tested for readability. We are not testing the participant’s memory or reading skills. If a test participant cannot find the requested information, or not quickly enough, in most cases there is something wrong with the leaflet and not with the tester.
Test participants are interviewed by a person with considerable experience in interviewing patients, and who has good listening and observational skills. The interviewer must be very familiar with the contents of the leaflet and must closely cooperate with the writer of the leaflet, or with the person responsible for the revisions.
The test questions must concern the most important usage and safety aspects of the leaflet. The sequence of these questions has to be randomized so that the requested information will not be found in the leaflet in the same order; participants have to search in different sections of the leaflet and have to look on both sides.
Please check service offers about readablity testing at Outsourcing Network