Some General Remarks on Consulting

by D.R.Cox

This article first appeared in Liaison 13.1, February 1999.

A few general comments on statistical consulting are given below. By “investigator” is meant the person or persons who are working in the subject-matter aspects of the study involved. To some extent the points are counsels of perfection and have to be applied in the light of constraints such as on time and employment conditions. The points are worded in terms of analysis of data; minor modifications are needed if design rather than analysis is involved.

  1. If possible collaborate (i.e., work with an investigator over a period of time) rather than consult (i.e., some occasional discussion of very specific statistical issues with the investigator).
  2. Be interested in the subject matter involved.
  3. Aim to use the terminology of the subject matter field where it differs from common statistical usage.
  4. If on reflection the investigation seems misguided, retreat from the consultation as soon as politeness and practicality permit.
  5. If collaborating, go to subject matter seminars from time to time, and read journals in the field.
  6. Discreetly determine how much understanding of statistical issues the investigator has. Mechanical use of significance tests to confirm overwhelming effects for example is a bad sign.
  7. Frequently review what is being done to check that the statistical analysis addresses the correct questions. This may help the investigator clarify thinking as well as protect against the most common error in statistical work – answering the wrong question.
  8. Aim, if feasible, to see some raw data, to understand the measurement processes involved, and to have some appreciation of the general quality of the data.
  9. Enquire into aspects of the study design that might have a bearing on the appropriate analysis.
  10. Begin with very simple methods.
  11. If possible, end with simple methods.
  12. Since nice ideas for analysis often do not work the first time, be prepared to do modifications.
  13. Do not be frightened to make strong assumptions. When a preliminary answer has been obtained then consider which of the assumptions made might be crucial.
  14. Take considerable care over presentation of conclusions.
  15. If your work is to be acknowledged in a paper or a report, ask firmly to see what is written before it is submitted.
  16. If you feel you should have been a co-author and have not been invited to be, pause for a few days. If, on reflection, you still feel the same, speak quietly to the friendliest of the investigators pointing out, assuming it is true, that you have spent a lot of time and thought on the work.
  17. Occasionally; very rarely one hopes; be prepared to say that the data are incapable of throwing useful light on the issues involved.
  18. Find a good balance between thinking things out for yourself and obtaining advice from statistical colleagues (and, of course, therefore, finding time to help them in return).
  19. If more than ten per cent of what you do ends up by being directly useful, you are doing well.
  20. If the investigator begins by saying he has a trivial little problem which he is sure you will be able to sort out immediately, don’t altogether believe him!

Have I always followed these points myself? No.
Do I wish that I had? On balance, yes, probably.

Support from a Leverhulme Emeritus Fellowship is gratefully acknowledged.

Nuffield College and
Department of Statistics,