Summary

A written protocol is vital before conducting research, and should consist of sections on preliminary comments, subjects and methods, details of interventions and outcome measures, and methods of analysis. The process of developing the protocol provides a disciplined consideration of all aspects of the study.

Key words

Introduction

The process of protocol development is also an important preparation for both ethics committee approval and grant application. Although both Local Research Ethics Committees (LRECs) and grant-awarding bodies will have their own detailed forms to complete, they will also want to see the formal protocol, and all the questions they may raise should have been considered already. Both these bodies will be interested basically in whether the researcher has defined a specific and important research question and has a good chance of answering it.

Some general guidelines for developing protocols follow, with the emphasis on controlled trials. No single set of guidelines can cover all circumstances but the general principles should be applicable. The very highest standards for designing, implementing and reporting research into drugs are contained in a European Union document entitled Good Clinical Practice (1) which aims to reassure the public that the data generated are credible and that the rights, integrity and confidentiality of subjects are protected. Good clinical practice involves detailed monitoring for quality assurance at every stage of the research, although in practice most clinical projects do not have the funds to pay for monitoring to this level. However, simply knowing about the highest available standards is likely to improve any researcher’s skill at developing a protocol.

Sections of the protocol

Preliminary comments
The research team is named, and the roles of the investigators, clinicians, research nurses, evaluators, and statistical advisers are set out. Usually, this is the opportunity to establish authorship of the final report, not only whose names are included in the list of authors, but also in which order. Grant-awarding bodies will need to know that the individuals responsible for different roles have the necessary skills and resources, and may ask for Curricula Vitae. This is also the place to define precisely which organisation or individual has overall responsibility for the research, and to describe the setting for various parts of the project. Any sponsor should also be disclosed.

Introduction or background
The aim of this section is to leave readers in no doubt about the need for this research. By the end of the introduction they should be able to conclude that the researcher has done a thorough literature search, has read all relevant papers about previous research, and is fully briefed about both the condition and the proposed treatment to be investigated. This section then concludes with a clear statement of the study objectives.

The Introduction should be as succinct as possible. Its usual structure consists, first, of introducing the medical condition, commenting on its frequency and its clinical or economic importance, and describing the current management, perhaps pointing out where conventional treatment is inadequate. Second, it should explain the possible role of acupuncture in treating the condition, or perhaps similar conditions; in designing a controlled trial, there needs to be a reasonable expectation that acupuncture will help patients, but not enough evidence to conclude that acupuncture is definitely effective for the condition. This is usually expressed as there being sufficient equipoise in clinical opinion.

Indication that a complete literature search has been done is necessary in the introduction, and essential references should be quoted to support each step of the argument. In Denmark, there is an even more demanding condition for ethical approval: the investigator should have first carried out a full systematic review on the topic! (2).

Finally, the text of the Introduction should be appropriate to act as the introduction to the final report or paper. There is a tendency for authors to use the introduction to display their extensive knowledge about the subject, but this is largely counterproductive. Readers need to learn the essential arguments, logically and clearly set out, without extraneous and baseless supposition.

Aim or research question
The Introduction will end in a general statement of the study objectives, e.g. "to determine whether acupuncture is effective in rehabilitation after stroke". The next section defines precisely the specific objective(s) of the study. A clearly defined, single main question is the most important requirement for successful clinical trials. This can be expressed as either a question or an hypothesis (preferred in the USA). Thus, for example: "Is standard manual acupuncture superior to sham acupuncture in improving the quality of life after ischaemic stroke?" is equivalent to the hypothesis "Acupuncture is superior to sham acupuncture in improving... ". The aim is sometimes stated as a null hypothesis, for example "Acupuncture is not superior to sham acupuncture...": because this is what inferential statistics actually test, although this can just add confusion.

The researcher should be clear whether the objective of the study is to test the efficacy or the effectiveness of acupuncture. Efficacy is the extent to which acupuncture produces a beneficial result under ideal conditions, whereas effectiveness is a measure of the success of acupuncture when carried out in an average clinical environment. Efficacy studies have much tighter inclusion criteria, whereas effectiveness studies have more clinically relevant outcome measures.

Secondary questions may be asked in the trial, but should remain subordinate to the main question. However, they must be defined with just as much rigour and precision.

Design
A single brief sentence makes it clear to the reader what you are planning - and will make it instantly obvious whether the design is right! Most rigorous studies follow randomised subjects forward over time, and may have various levels of blinding, in a design with 2 or more parallel arms, expressed from the following options:

1. Prospective, retrospective (theoretically)
2. Randomised, non-randomised
3. Blinding of: subject, evaluator, experimenter
4. Control group: sham, no treatment, comparative treatment
5. Number of parallel arms

Subjects and methods

Sample size
Calculation of sample size is an essential component of research design that is all too often omitted. The purpose of this is to recruit enough subjects to get valid results but not more than is needed to produce an answer: so as not to waste the time, money and effort of participants and investigator. The required sample size depends on four variables:

alpha, the threshold for significance (which is typically set to 0.05)
beta, the probability of missing an important difference or making a Type II error (typically, p=0.20, which is equivalent to 80% power)
delta, the minimum difference in the primary outcome measure between groups that you wish to detect as statistically significant (considering clinical significance too, if appropriate)
SD,the estimated standard deviation of the primary outcome measure, which can be gained from a pilot study or from previous publications.

Altman gives an excellent description of a simple method (3).

Inclusion and exclusion criteria
These criteria are applied to make sure that each participant is appropriate and the group as a whole is homogeneous. The inclusion criteria will usually be a more precise statement of the medical status, such as the type and severity of a headache. Importantly, it is wise to include informed consent as one of the inclusion criteria, to give it due prominence.

Exclusion criteria may include certain medical conditions (diagnosis of cancer or use of particular drugs, for example), contraindications to acupuncture (such as refusal to be needled, or major bleeding disorder), previous acupuncture treatment (for a trial that involves participant-blinding), and anything that will interfere with measuring the effect of the therapy, including lack of necessary skills (such as the ability to understand instructions on how to complete a questionnaire), the likelihood of getting better without the treatment (such as those with very mild stroke), and the use of other simultaneous treatments.

Information sheet
Once these criteria have been applied, full verbal and written information about the trial is given, questions are then invited, and answers given. This 3-part procedure is essential for informed consent. The information sheet and consent form are an integral part of the protocol, as an appendix. Many LRECs give clear advice on what should be in an information sheet, and many also produce a standardised consent form, When consent has been given, the patient becomes a participant and is formally enrolled in the study, receiving a unique study number.

Withdrawal criteria
Data from participants who simply drop out of the study must still be included in an intention-to-treat analysis, discussed below. Those who withdraw for genuine reasons can be excluded, provided the rules for withdrawal have been defined and are applied without bias. For example, "Participants developing serious intercurrent illness may be withdrawn from the study after assessment by a physician not involved in the study and blinded to the participant’s group allocation"! Group allocation
If the participants are to be divided into groups, specify the allocation method. This should usually be by randomisation. Include information on how randomisation is achieved, and how the allocation will be concealed at the time when the decision is taken to include the participant in the study. It is possible that participants need to be stratified by prognostic factor before randomisation.

Intervention and control procedures

Control intervention
The control group may receive nothing, an active therapy, or a sham form of acupuncture, depending on the research question. For a discussion of the possible controls that have been used in acupuncture trials, see Filshie and Cummings (4). State full details of the control intervention so that it, too, can be replicated.

Schedule of interventions
State the frequency and number of treatment and control sessions. lt is usual to specify what proportion of the total number each participant must complete to satisfy the protocol, and whether gaps in treatment are allowed, e.g. for holidays.

Blinding of participants and evaluators
If the design involves blinding, state how this is to be achieved and maintained and, moreover, state how it will be checked at the end of the study. If a sham control is used, consider testing whether participants have been able to tell that it is not genuine acupuncture. This will involve asking standard questions of both the active and the control groups, since they should be managed in an identical way as far as possible.

Patient evaluation

Outcome measures
This should be a single measure, the primary outcome measure, on which the overall success or failure of the treatment is assessed. Objective measures, such as blood pressure, are ideal for efficacy studies where the aim is to determine whether acupuncture has an effect on the condition at all. When studying the effectiveness in real life, however, it may be more appropriate to use a measure that is more relevant subjectively to the participants, such as pain or quality of life. lt is essential to use a primary outcome measure that has been validated, and to quote the reference.

There are other measures, known as secondary outcome measures, which support or amplify the information from the primary measure or answer secondary questions. For example, if the primary outcome measure in a study of stroke is the Activities of Daily Living, secondary measures could include measures of spasticity, timed lo-metre walk, dysphagia, and Hospital Anxiety and Depression scale. Purpose-designed measures may also be used here, and may throw useful light on other aspect of the treatment effects that the researcher is interested in.

Data collection
Baseline data will be required from all participants in order to describe and compare the make-up of the experimental and control groups. Baseline data should include usual demographic details plus any information that may affect the outcome in individual participants. For example, in a study of smoking cessation, patients who live alone have less success in stopping, so this risk factor needs to be documented. The actual data forms should be added to the protocol as an appendix, including the standard forms for recognised measures and those that have been developed specifically for the study. Follow-up data may be of crucial importance and the schedule for collecting it must be specified.

Economic evaluation
It has been recommended that economic evaluation should take place alongside clinical evaluation in all clinical trials, so that some evaluation of the cost-effectiveness of the therapy may be obtained. If measurement of costs is to be included, these need to be predefined in the same way as other outcome measures.

Adverse reactions
Every study should make the collection of adverse reactions a routine. Participants should be asked specifically about adverse events.

Confidentiality
This is a good point to describe how personal information is to be kept confidential. This might involve training staff, keeping paper records securely locked up, and removing personal details before entering data on computers. The organisation’s Data Protection Officer may need to be notified, if appropriate.

Ethics
The protocol may contain a statement of any particular ethical problems which are anticipated (for example, obtaining consent from patients who have had a stroke affecting the speech area), any sources of ethical advice, and the relevant LREC. Ethics committees are primarily concerned with the interests of the participants.

Statistical methods

Analysis
Analysis usually involves statistical tests, and the choice of test depends on the answers to two questions: What kind of data will be collected? and What is the goal of the study? Different types of data include measurement from Gaussian population, rank, score, or measurement from non-Gaussian population, binomial or dichotomous data, and survival time. Examples of a goal are to compare two paired groups, or to compare three or more unpaired groups. Precise statistical methods are beyond the scope of this article, but Motulsky gives a helpful table (5). It is usual to include the words intention-to-treat analysis and to include all data including that from those who dropped out of the study.

Miscellaneous

Time schedule and feasibility
The proposed time schedule should be outlined, marking milestones such as ethics approval, preparation of forms, staff training, patient enrolment, data collection and data entry, analysis, and writing the report. Attention may need to be paid to precisely how the data forms are to be processed and the data extracted. This section involves also assessing the feasibility of achieving each step within. the allotted time.

Conclusion

Acknowledgement
The authors wish to acknowledge the help of Prof Ernst in the preparation of this article.

Adrian White MA BM BCh
Jongbae Park MKM KMD
Department of Complementary Medicine
School of Postgraduate Medicine
and Health Sciences
University of Exeter, 25 Victoria Park Road
Exeter EX2 4NT (UK)
Email: a.r.white@ex.ac.uk

References

1. Committee for Proprietary Medicinal Products (CPMP) Working Party on Efficacy of Medicinal Products. Good clinical practice for trials on medicinal products in the European Community: EEC notes for guidance. Pharmacology & Toxicology 1990; 67: 361-72.
2. Editorial. Denmark takes a lead on research ethics. British Medical Journal 1998; 316: 1189.
3. Altman DG. Practical Statistics for Medical Research. 1st ed. London: Chapman and Hall; 1991. p.455-60.
4. Filshie J, Cummings TM. Western medical acupuncture. In: Ernst E, White AR, editors. Acupuncture: A Scientific Appraisal. Oxford: Butterworth-Heinemann; 1999.
5. Motulsky H. Intuitive Biostatistics. Oxford: Oxford University Press 1995; p.298.

You may subscribe to the paper or electronic version of the Journal. Copies of back issues can be purchased, subject to availability. Details of current prices are available here. Material for publication is welcome. Please read our advice for authors.

For subscription or advertising enquiries, please contact:
The BMAS Administrator, Newton House, Newton Lane,
Lower Whitley, Warrington, Cheshire, WA4 4JA
Tel 01925 730727 Fax 01925 730492
E-mail: Admin@medical-acupuncture.org.uk