Author: Sam Goldstein PhD & Dr. Barbara Ingersoll
Source: Excerpts from an article by Sam Goldstein PhD & Dr. Barbara Ingersoll, C.H.A.D.D.E.R., Fall/Winter, 1992

How Are New Treatments Evaluated?

The road by which a particular treatment is shown to be effective can be long and arduous. The process begins with the formulation of a hypothesis, or idea. This hypothesis is usually based upon an existing body of knowledge (e.g. “Since stimulant medication helps ADHD children, it might also help ADHD adults”).

The second step is the development of a protocol to evaluate the effectiveness of the proposed treatment. The treatment itself, and the way in which it will be implemented must be carefully defined (e.g. “X amount of medication will be provided to Y number of adults with histories of ADHD for Z period of time”).

The researcher must also specify the way in which the effectiveness of the treatment will be evaluated. Care must be taken, for example, to be sure that the effects of the treatment are not due simply to placebo effect. Placebo, which is Latin for “I shall please,” refers to the very well documented fact that people may respond to all sorts of ineffective treatments as long as they believe that the treatment has the power to help them. Placebo effects can be more dramatic that most people realize. In a classic example, a drug known to cause vomiting actually brought relief to people suffering from severe nausea and vomiting when they were told it would help them.

The researcher must also take care that all who participate – researchers and research subjects alike – are “blind” (unaware of) whether they receive the active treatment or the placebo treatment. Otherwise, the expectations of either party could influence the findings.

Appropriate measurement techniques and statistical tests must be built in, so that the entire scientific community can evaluate the findings. Finally, the results must be subjected to the scrutiny of this group. This means that the findings must be published in journals which accept articles for publication only after careful examination by other scientists who have expertise in the particular field.

As if this were not enough, findings are not considered substantive until additional studies have been conducted to reaffirm (or disconfirm) the findings. This process can take years but it allows us to make sound decisions about new treatments.

Alternative Treatments: Another Path

There is also a second path which some practitioners follow, sometimes in an effort to short cut the longer, more accepted process. This path is fraught with many problems. On this path, proposed treatments stem from concepts which are outside of the mainstream of existing knowledge. They may be instituted long before there is any research which supports their effectiveness often after only brief, poorly designed trials involving a small number of people. Measurement techniques and statistical means of evaluation are scanty, at best, and often single case studies are offered as “proof” of the effectiveness of the treatment.

This treatment approach is usually publicized in books or journals which do not require independent review of the material by recognized experts in the field. Often, in fact, the advocate of a particular treatment approach publishes the work. This fact should raise the warning for the consumer parent. Additionally, although parent support groups have an essential role in the treatment of childhood disorder, in the case of a controversial treatment, parent support groups advocate one and only one treatment. These groups then play an important role in publicizing and promoting that treatment. Unfortunately, enthusiasm is not a substitute for careful scientific investigation.

These alternative interventions commonly claim effectiveness for a broad range of problems. When asked for proof to support these claims, however, proponents are unable to produce documentation. Proponents may also claim to have access to knowledge and information not shared by the medical community at large and, when their treatments are criticized, they may explain this as reflecting a conspiracy against them in the scientific community.

How Can A Parent Be A Wise Consumer?

If you are the parent of a child with ADHD and/or learning disabilities, you know how difficult your job can be. Of course, you want to obtain the very best treatment for your child. In a spirit of “How can it hurt to try it?” you might be tempted to throw caution to the wind when you hear about a new treatment that promises to help.

But promises are not enough. You also have the responsibility to invest your family’s resources of time, money and energy wisely. This means that you must become an informed consumer.

In this paper, we have provided general guidelines for evaluating new treatments. Listed below are some additional tips to help you recognize treatments that are questionable.

  • Overstatement and exaggerated claims are “red flags”. Be suspicious of any product or treatment that is described as “astonishing,” “miraculous,” or “an amazing breakthrough”. Legitimate health professionals do not use words like these, nor do they boast of their success in treating huge numbers of patients.
  • Be suspicious, too, of any treatment that claims to treat a wide variety of ailments. Common sense tells us that the more grandiose the claim, the less likely it is that there is any real merit behind it.
  • Do not rely on testimonials from people who say they have been helped by the product or the treatment. Enthusiasm is not substitute for evidence and legitimate health professionals do not solicit testimonials from their patients.
  • Be sceptical about claims that a treatment is being suppressed or unfairly attacked by the medical establishment. Legitimate health professionals eagerly welcome new knowledge and better methods of treatment for their patients. They have no reason to suppress or oppose promising new approaches.

Excerpts from an article by Sam Goldstein PhD & Dr. Barbara Ingersoll, C.H.A.D.D.E.R., Fall/Winter, 1992