The first article on how to write up a QIA e-portfolio entry was used to explain the theory and practice of writing QIAs.
This second article explores how a GP trainer assessed the QIA and awarded a grade and it should be read in conjunction with the first article here.
Project title and why it was chosen
The trainee explained that clinician grumbles about ‘rambling’ meetings and stress when their clinics after the meeting started late, sparked the project. This obvious frustration with an area of clinical practice triggered the project. As the title and reasons for choosing the QIA are clear and are based on an identified practice need, the trainee scores ‘above expectations’ (AE).
However, while there is reference to three sets of appropriate guidance relating to meeting facilitation and feedback, the guidance is not considered in any detail, so the trainee scores ‘meets expectations’ (ME).
In attempting to finish meetings on time so clinics can start on time, there is consideration of the impact of the QIA on patients, so the trainee scores ‘meets expectations’ (ME).
The trainee discussed their goal by defining what ‘focused’, ‘useful, and ‘relevant’ meetings looked like, but did not specify a clear time frame. The trainee scores ‘meets expectations’ (ME). Had the trainee summarised their aims in a SMART (Specific, Measurable, Achievable, Relevant and Time defined) format they would have scored more highly.
By writing ‘I will know that the changes I made to the meetings are an improvement if feedback is positive and we ended on time, allowing clinicians to start clinics on time’, the trainee has considered what an improvement looks like. The trainee scores ‘meets expectations’ (ME). By discussing the clinics starting on time after the meeting, there is a connection with patient safety and/or patient care. The trainee scores ‘meets expectations’ (ME).
Describe what baseline data or information you gathered
The trainee collected baseline data from their first meeting. They should have collected data from the ‘old style’ meetings. Had this data been presented to us, we could judge what improvements are needed.
The trainee tells us some data collected by talking to attendees (there were grumblings), but a formal survey was not done. As insufficient information is provided to demonstrate the ‘problem’ was fully understood prior to the improvement being designed and implemented, the trainee scores ‘below expectations’ (BE).
The trainee did identify suitable QI tools namely conversation, followed by conducting a survey, and using thematic analysis. The data gathered was qualitative.
However, it seems the survey gathered feedback about the quality of the facilitation style and asked about what could be done to improve facilitation. The focus of the survey was incorrect. We need data on whether participants found meetings more focused, relevant, useful and running to time. The trainee scores ‘below expectations’ (BE).
The data was not presented well. We do not know how many conversations were had. We do know two out of six people completed the first questionnaire. We do not know if they scored the meetings on a five-point scale and what the scores were. We don’t know if the meeting ran time and whether the clinics started on time. The trainee scores ‘below expectations’ (BE).
Describe what subsequent data or information you gathered
Once again, the evidence provided is about how well the meeting was facilitated and what could be done to improve facilitation. As the data cannot tell us if the meeting was more useful, relevant, focused and ran to time, is not capable of demonstrating the changes the trainee wanted to make.
The trainee’s choice of data is incorrect, and the manner in which the data is presented is poor, so they score ‘below expectations’ (BE).
How did you plan and test out your changes?
While there is a clear and appropriate use of a PDSA cycles in the planning and implementation of the project, the data being gathered is about the quality of facilitation in the meetings. It seems the trainee adapted a presenter’s feedback form without much thought to adapting it to make it fit for the purposes of the QIA.
The trainee needs to revise their survey form to collect data about whether team members found the meetings to be relevant, focused, useful and running to time.
How have you engaged the team, patients and other stakeholders throughout the project?
By discussing the conversations and feedback forms, the trainee provides a description of how different stakeholders were engaged. However, the trainee forgot to collect data about how long the meetings took and whether clinics started late.
To score more highly, the trainee could collect data from patients in the follow-on clinics about their waiting times and their levels of satisfaction, and what actions the practice took to inform them about waiting times, and how they felt about being kept in the loop.
The trainee does not tell us how they intend to engage with clinicians who do not complete the survey form. Therefore, there is inadequate reflection on the challenges of engaging different team members with the survey, and there is no action plan on how collecting feedback will be improved. The trainee scores ‘below expectations’ (BE).
Summarise the changes as a result of your work and how these will be maintained
Had the focus of this QIA been on improving facilitation or chairing skills, then the trainee would have scored well because they produce a summary of changes that is clear and appropriate to good meeting facilitation.
They also discuss steps to enable maintenance of the changes to their style of facilitation. However, the trainee scores ‘below expectations’ (BE) because the survey form is not correcting the correct data so there is no evidence that the changes made will prevent future meetings from rambling on and running over time.
The facilitation style of the trainee may be improving, but there is no evidence that change has been embedded by the organisation. The trainee scores ‘below expectations’ (BE).
What have you learnt and have you got any outstanding learning needs?
The trainee reflected on their facilitation style and identified appropriate learning needs. However, the trainee’s choice of tools to gather and display evidence of change was sub-optimal. This can be remedied by changing the survey form to collect more appropriate data about the meetings; collecting data on the time the meetings started and ended; whether follow-on clinics started late; and the views of patients in these late-running clinics.
If the trainee collects better data, by designing better data collection tools, and presents the data properly, we would be better placed to see if the patient experience in follow-on clinics was improved or if clinicians, now starting their clinics of time, were less stressed and less likely to make errors. The trainee scores ‘below expectations’ (BE).
I hope that these two articles on writing and assessing a QIA help you to develop your understanding of the topic. In summary, QIA is about data driving change. That change should improve the patient experience and patient safety.
So, pick a project that focuses on making patients safer; collect baseline data (the data that shows us the situation before you make any changes); use the right tool(s) to collect data; present the data simply; describe the small changes you make; and collect and present data showing the impact of those changes. Finally, reflect on the experience.
- Dr Prashini Naidoo is a GP trainer in Oxford. She has written three books on how to pass the CSA. The latest book CSA Practice Cases for the MRCGP was published in January 2016.