Patient Surveys

As a patient advisor I have been asked to review a provincial client ( patient ) experience survey.

The survey consisted of 19 questions where clients rated their experience on a scale from 1 to 10 with one bing the lowest. The three client experiences measured were 1) overall patient experience 2) being consulted in decision making in their health care and health service 3) culturally sensitive care. I have copied part of the measures for your input.

Overall, the combined scores for each measure represent high levels of positivity, with two out of the three measures
exceeding the target of 90% from 10796 completed surveys.

Results from the Patient Experience Survey have been shared with Teams, Committees and Councils to inform
targeted action plans to address areas for continued quality improvement. Numerous initiatives are currently underway
to enhance patient experience such as:

 Broad initiatives are in place to include patients and family on provincial Teams, Committees and Councils to ensure
that the patient voice is heard, and used to improve overall experience.’
 Province has established a patient, family and public advisory council, a joint council for patients, family and public
representation to engage and plan together to enhance the experience of care.
 Patient Feedback Line has been developed, it is a 1-800 number that patients and families can use, and it is directly linked to designated staff in all zones.
 A ‘Patient Voice’ initiative is in development. This will see audio clips of patient stories available on province intranet for use by staff, management and physicians as an additional means of integrating patient experience
into planning and decision making.
 In Eastern Zone, electronic Patient Experience Kiosks will be set up in three hospitals to allow the provision of
real time patient experience feedback.

Overall patient experience score 89.5

Consulted in decision-making in their care and health service 92. 3

Culturally sensitive care 94.8

As I think this is great I wonder is this a real demonstration of our care. This is high and fabulous but I continue to talk to people who do not have a great experience. We have a shortage of doctors and huge wait list.

In your experience can this be valid and where would any of you suggest for improvement. In my experience I hung in the health care system for years. Where someone does not know what is wrong with you they shuffle you around and through my support group I still see the same.

I meet Wednesdays with the team. What are your thoughts?

Thank you

Virginia

Thanks Susan for your input. The method used was :" Results were calculates by deriving a composite score. Individual indicator scores were calculated as the number of positive responses divided by the total number of valid responses within each indicator. If there was more than one indicator per measure within a program area, a straight average was calculated of the two. The overall score for each strategic plan measure is an average across the six program areas. "

all areas results were similar.

I was wondering if I am the only one that felt this way too.

According to the survey the scores where measured from 1. Acute care -inpatient 2. acute care-ambulatory 3. Mental health & addictions 4. cancer care 5. Long term care and 6. Primary Health care. I will look more into this Wednesday on how administered.

Susan anything else I need to inquire about.

All input is appreciated.

Thank you

Virginia

Virginia,

Thanks. Can you send me a copy of the questionnaire?

How it was administered would be things like did they send it by email? By mail? if so, what was the response rate? Based on the large number of responses, I doubt that they administered it in person.

The methodology you have given is the methodology for calculating the results based on the responses they got on the questionnaires. This would have been done by specialists in statistics and analysis.

But I simply cannot believe these results reflect the actual opinions and experiences of people. 90% positivity is virtually perfect in the scheme of things. There is something wrong. That would be my feedback to them: "Your numbers are too good. There's a problem here. You need to take a closer look at your whole survey process starting with the questions."

I'm not an expert is survey research, not at all. But I just can't believe these results.

Susan

Susan

Yes I can send but would prefer to send via private email.

Virginia

This is list of just the questions asked. I wonder how many where LGQT or African American or/ and Indigenous with the culture questions.
OVERALL ASSESSMENT
Q20. Using any number from 0 to 10, ...what number would you use to rate this clinic, program or service?
Q21. Would you recommend this clinic, program or service to your family and friends?
RESPECTING CLIENT VALUES, NEEDS AND PREFERENCES
Q2. How good was the health care provider at listening to you?
Q4. How good was the health care provider at involving you in decisions about your care?
Q6. Did your health care provider discuss with you your main goals in caring for your health?
CULTURAL VALUES AND DIVERSITY
Q9. Staff took my cultural values and those of your family or caregiver into account?
Q10. I feel that my diversity status and that of my family/caregiver, was respected and valued by staff?
*Q10 - Those who stated that they did not know, did not remember or that this issue is not applicable were excluded from the analysis.
SHARING INFORMATION, COMMUNICATION AND EDUCATION
Q3. How good was the health care provider explaining tests and treatment
Q7. Did your health care provider ask if you felt you could do the recommended treatment plan?
Q8. Does your health provider tell you about side effect you might get from a medicine?
Q18. How well do you understand the nature and causes of your health problems?
*Those who selected "I did not have any treatment," "I don't take any medicines" or "I don't have any health problems" were excluded from the analysis of Q7, Q8, and Q18 respectively.
COORDINATING AND INTEGRATING SERVICES ACROSS BOUNDARIES
Q1. How easy was it to get through to someone at your clinic on the phone?
Q5. How helpful did you find the front office staff?
Q12. Were there times when you had difficulty getting the health care or advice you needed?
Q16. Were you encouraged to go to a specific group/program or class to help you mange your health concerns?
*The 'NO' responses in Q12 were counted as positive for the purposes of calculating an overall score for this dimension.
*Responses of "Not Applicable" in Q1 and "NO, I haven't needed such support" in Q16 were excluded from the analysis.
ENHANCING QUALITY IN LIFE IN THE CARE ENVIRONMENT AND IN ACTIVITIES OF DAILY LIVING
Q13. Did the health care team here help you feel that your everyday activities such as diet and lifestyle make a difference in your health?
Q14. Did the health care team here help you feel that your everyday activities such as diet and lifestyle make a difference in your health?
Q15. Does the health care team here help you feel confident about your ability to take care of your health?
Q19. How confident are you that you maintain changes in your health habits like diet and exercise, even during times of stress?
*Respondents who answered "Does not apply to me" in Q14 were exluded from the analysis of that item.

Virginia

These results are too good to be true, literally. I'm having a REALLY hard time believing they are valid.

What province is it?

How was the survey administered? To whom? What methodology did they use?

In most market research it is considered normal for responders with the most negative expeience to be the most likely to respond.

However, questions raised about the validity of the patient survey administration/distribution plan are well taken. tho sometimes difficult to do, the best healthcare survey projects make some attempt to explain administration/distribution methods to include any or all of the following concerns & perhaps more:

- # of patients 'eligible' to take a survey over a specific time span (time of day/weeks/months) then calculate a % of those completing a survey over those patients who cycled through the site or were 'exposed' to an invitation to participate post treatment.

- distribution/recruitment methods section can be very detailed introducting challenges to any publication/report writeup. healthcare publication/reports do favor positive results/outcomes (publication bias). those patients with both time and having had a positive experience can be more likely to participate in a patient survey in many situations. this creates an inherent bias in response & while it should be addressed in the final publication/report it may not be fully addressed because of word count limits (sort of like my overlong messages here however, lead investigators are still responsible to explain their attempts to optimize the # of patient responders OR alternately address those details as requested.

- patients who may still be in pain, upset, or confused may be less likely to participate in a patient survey (or more likely) because depending on their perceived vulnerabilities.

- some researchers make a research assistant available to engage in more careful recruitment techniques. study costs often cause researchers to skip more labour intensive recruitment methods which may result in omitting this step in patient surveys in a healthcare settings.

- if survey followup is done too long after a healthcare event the details may be subject to recall bias (on part of the patient). e.g. 6-12 weeks post treatment experience is often used as a maximum benchmark in assessing recall bias.

- easy auto-response techniques (like a secure website in the waiting room or hospital room w-locally availalbe tech support on hand) can work well if properly supported by the study design.

sorry for another long 'ish' comment.

anita l-l

Virginia, my email address is: susan@myalzheimersstory.com

One thing i could do is to "replicate" the survey with the questions rephrased/reconstructed and see if we get similar results. I would use survey monkey to do this, but i couldn't do it until the end of the week, and of course it would take time to get responses, but it would be an interesting exercise. The way that questions are asked and also the factor that is rated on the scale may make a difference in the results.

The other thing that's important to know is what the rating scale was exactly; you say 0 was the "lowest," but lowest what? Was it O is "negative" and 10 is "positive?" Or 0 is "not satisfied" and 10 is "extremely satisfied?"

If you have a copy of the survey, please email it to me.

Susan

I agree with all you've written @Anita, and all of that said, do you think the results of this survey likely reflect real patient experience?

My gut tells me "no way!" But maybe it's just me...

Susan

truth be told, Susan. it is hard to say until more details on distribution techniques are explained to you.

If the 1st iteration of the survey were "pre-tested" as part of the study design then it is likely it does reflect the patient experience for those who actually completed the survey.

If it was not 'pre-tested' your idea to duplicate in a "similar setting" would be a good idea as long as no substantial changes are made to the survey the 2nd time around.

info on 'pre-testing' is readily available.

all good luck & hats off to you for doing this.

anita l-l

Anita, i think you've hit the nail on the head when you say it may "reflect the patient experience for those who actually completed the survey."

However, I just can't believe it would reflect the overall experience of the total patient population across an entire province. 90% of patients have a positive experience of health care providers? Honestly, I almost spit my tea all over my keyboard!

Then again, I have been proven wrong in the past - in July 1994, I think it was...

Susan

Susan, my gut says the same and I am also in healthcare. I find this hard to believe, really 90 % !!

I appreciate the input and will email you with what I have.

Virginia

Susan

Email is sent

Virginia

On the last OECD survey with people over 65 across the normally included countries, I was surprised at the % of people who were positive about being included in their care plan. I have never had that happen.

I agree with you all that those results sound too good to be true.

In my market research experience I developed the strong feeling that the person who composes the questions has a lot of influence on the answers

Annette

Patient Surveys 4 14 17

Patient Surveys

4

14

17