Home > Frequently Asked Questions
Frequently Asked Questions
How do I log-in?
Only deanery staff and working group members have a log-in. Click on the 'log-in' link at the top of the page and enter your email address and password. Your password will have been emailed to you from a PMETB email account.
How do I find out when the site is updated?
To be kept informed of changes to the site, please register using the form provided, accessed via the link at the top of the page. Deanery staff who have log-ins are already registered.
How were the items developed?
The Trainee Survey
The generic survey items from the 2006 National Survey of Trainees were used again in the 2007 survey. For full details of their development please see the:
2006 National Report (link).
Foundation and specialty specific items, constructed in response to comments from last years survey to measure adequate experience, were submitted by the colleges/faculties and placed on a web form for further consultation. Where possible members of the trainee survey working group adapted the items to fit the agreed response format:
How many (cases/procedures) have you performed?
- In the last 4 weeks (number required)
- In your career to date (number required – estimate)
- I am confident that I have gained enough experience for my current stage of training (Likert Scale)
SAC chairs or their equivalent in each specialty or sub-specialty were asked to review the items and approve them for their inclusion in the survey.
Trainer survey
The survey items were generated by PMETB’s survey working group and in conjunction with a literature search. A web consultation was conducted and PMETB commissioned a face validity study that was conducted by researchers in the Northern Deanery and National Education for Scotland.
When were the surveys conducted?
Trainee Survey
The pilot of the administration process was conducted in the Wales deanery from 15 November 2007; all other deaneries started from 4 December 2007. The survey closed on the 28 February to allow time to send reminders to non-respondents and obtain missing deanery data. This year the survey was mandatory for all run-through trainees, as stated in the Gold Guide :
7.36 “[doctors] must take part in systems of quality assurance and quality improvement in their clinical work and training (e.g. by responding to requests for feedback on the quality of training, such as the National Trainee Survey)”
For SpR doctors, COPMeD have agreed that proof of survey completion will be required for RITA sign-off.
Trainer Survey
The pilot of the administration process was conducted in the London deanery from the 14 December 2007, all other deaneries commenced from 3 January 2008. The survey closed on the 28 February 2008.
Did the surveys have the required approvals?
Following the National Research Ethics Guidance Differentiating Audit, Service Evaluation and Research ethics approval is not required as the surveys are classified as service evaluation. This classification of the work as service evaluation has been confirmed by the Nottingham Research Ethics Committee.
Both surveys received ROCR-Lite (sic) approval:
- Trainee Survey: ROCR-Lite/OR/0214
- Trainer Survey: ROCR-Lite/OR/0215
How was the survey administered?
Trainee Survey
Trainees’ emails were collected from deaneries together with post data. Emails were sent via the PMETB database to trainees and reminders were sent to all non-respondents on up to 10 occasions (31% completed on the first mailing, a further 46% completed by the fourth mailing). Deaneries were informed of emails for which errors were received and where possible correct addresses were obtained.
Trainer Survey
Deaneries provided a nominated contact such as the director of medical education or equivalent within each of the local education providers within their area. Nominated contacts were then asked to cascade the survey invitation to all consultants at their location. Deaneries provided email addresses for active GP trainers and Foundation tutors, who received their email invitation directly from PMETB’s database.
What was the overall response rate?
The total number of usable responses for the National Survey of Trainees were 33,329. The response rate is 66% where denominator is all valid trainee emails obtained via deanery returns and the trainee opt-in form.
The National Survey of Trainers was administered via 2 routes. There was a total of 2,211 (45%) GP and Foundation tutor responses, the denominator is all valid trainer emails obtained via deanery returns and the trainer opt-in form. The total number of consultant responses was 7,796 (~20%) where the denominator is the workforce census data (2007 data for England and 2006 data for the other nations). This gives a total of 10,021 responses for the trainer survey.
How can I report errors on the site?
If you notice any errors on the site please click on the 'feedback' link at the left- hand side of the screen in which the error is located and describe the problem. The feedback form automatically captures the page you clicked from and so will help us identify the problem.
How are the grade groups defined?
Trainees in St2 and below are defined as lower, whereas trainees in St3 or SpR posts or above are defined as higher. Exceptions include those specialties where all trainees are defined as higher: chemical pathology, clinical genetics, clinical neurophysiology, histopathology, immunology, medical microbiology and virology, medical ophthalmology, nuclear medicine, occupational medicine, pharmaceutical medicine and public health medicine. Further exceptions include those specialties in which trainees in St3 and below are defined as lower and St4 and above are higher: child and adolescent psychiatry, clinical oncology, clinical radiology, forensic psychiatry, general psychiatry, old age psychiatry, paediatrics, psychiatry of learning disability, psychotherapy, community child health, core psychiatry training and all paediatric and psychiatry sub-specialties. For all specialties trainees in St grades who indicated they were in FTSTA posts were classified as FTSTA; trainees who indicated they were in FTSTA posts who indicated they were SpRs were classified as SpRs.
How are the specialty groups defined?
Specialties are group by college. Please see the Specialties table in the table downloads section of the site
How were the confidence intervals for proportions (%) calculated?
The confidence intervals of the proportion items (those indicated as percentages) were calculated using the recommended method from Altman, D.G., Machin, D. et al. Statistics with confidence 2nd edition; BMJ Books. 2000. The recommended method has better statistical properties than the traditional method, generates asymmetric values and can be used for any data.
How are the data used to improve the quality of training?
Indicator outliers are shown as triangles on the left hand side of the indicator charts. An outlier is defined as being below the national mean and in the lower quartile, shown as a red triangle or above the mean and in the upper quartile range, shown as green. This is reversed for the two bullying indicators. These outliers are areas that deaneries may target for further investigation locally; in which case the work will be documented in the deanery’s annual report, which deaneries are required to submit to PMETB, please see page 62 of PMETB’s
Quality Framework (link)
Can I receive a more detailed breakdown of responses?
Deanery staff will be able to receive more detailed data for their deanery via the Table downloads section of the site. These data are respondent level with all identifiers removed.
When will the National report be published?
The National reports for both surveys will be launched in the week commencing the 21 July.
Which report type should I be looking at when I'm prompted to select a group?
Reports available:
- Trainee Survey: specialty-groups within local education providers (GP by programme). This is most applicable to directors of medical education to view the results for their organisation. The report compares the results for each specialty group within the provider against the results for the given specialty group UK-wide. For instance Surgical Trainees at Health Hospital compared to surgical trainees UK-wide.
- Trainee Survey: specialty and sub-specialty grade groups within deaneries. This is most applicable to Training Programme Directors. An example of the data shown is cardiology trainees Higher (SpR and St3+, excluding FTSTA) within London deanery compared to medical trainees UK-wide.
- Trainee Survey: specialty and sub-specialty grade groups across the UK. This is most applicable to College and Faculty staff, as it shows a given specialty on a UK-wide basis compared to the experience of all trainees UK-wide. For instance Trauma and Orthopaedic trainees lower (St1 –St2, excluding FTSTA posts) compared to trainees UK-wide.
- Trainer Survey: All trainers (TPD excluded). This is most applicable to deanery staff. As it shows the results for all consultants across the selected deanery; please note that Training Programme Directors are reported separately. An example of the data shown is consultants in London deanery compared to Consultants UK-wide.
- Trainer Survey: All GP trainers and foundation tutors. This is most applicable to directors of GP training within deaneries. It shows the results for all active GP trainers and foundation tutors who responded, for instance GP trainers in London deanery compared to GP trainers UK-wide.
Where are Ns reported?
Ns are reported on the bottom level charts accessible by clicking through.
Are there instructions on using the site?
Yes these are in Reporting Tool Briefing Note 1 (link)
Can I access the National Survey of Trainers data by local education provider?
unfortunately, data are not reported at this level. Please note that due to response rate we are unable to report these data below the level of deanery.