Know-How Exchange : Research/Metrics

1. Typically, we would form a small team that is cross-functional to get different viewpoints.
2. That team would then review the responses for key words and phrases.
3. These key words/phrases would be placed on sticky pad notes and placed on the wall.
4. Work to group them into common themes.
5. Create a statement of "customer need" from each.
6. Do a quantitative survey with a broader group of "customers" to determine which one is the greatest need
7. Create projects in order of priority based on the results.

John

Dear Omarfdawood

What a time consuming problem! 56 open ended survey questions need some careful thought and John’s suggestions about using a team are certainly valid – one person can not do the analysis because their pre-conceptions and potential prejudice would produce a result of dubious validity. It’s in the methodology of analysis where I would differ somewhat.

Normative survey results (Unstructured subjective opinion) need to be turned into figures or quantified. Once you’ve done that the analysis is relatively straightforward.

The survey answers will tell you two basic types of information:

• Issues which concern the patients
• How strongly they feel about these issues

The first task, which I would tackle, would be to get a team of people, working in isolation to read though each survey and to write down the issues they feel are raised in each and every one. The issues need to be marked up on the survey form and highlighted – if this is on paper, a highlighter will do the job. If they are on a computer, use Word’s ability to monitor changes and highlight text. You need to do this so that you can go back to the issues and quantify the strength of feeling later.

These issues then need to be tabulated – again by each reader in isolation such that they totalise the number of issues which they have identified under each subject heading that they have created.

The team now needs to meet to discuss their lists and by referring back to the original marked-up survey answers they need to agree which of their “issues” are identical or near identical to each other, so that they can arrive at a total list of all issues.

Lastly in the quantification of issues, they need to sum them and report on the frequency of reporting of a given issue.

Now comes the hard part. The strength of opinion for each issue on each survey needs to be assessed. This is best done by the group working together. It is incredibly time consuming, but by using a scale of say 1 to 10 where 1 is a weakly expressed feeling and 10 is a vehement feeling, the issues can be individually graded in terms of strength of feeling.

In this instance, a geometric average is adequate to arrive at an overall strength of feeling for each issue. If you want to get a better picture of the range of opinions, you should do a histogram of responses for each subject topic and see how close to a bell curve for a normal distribution it is. If the results are much skewed, a more sophisticated form of statistical analysis might be called for.

In all honesty, the work involved in extracting anything which is more than just a subjective opinion from this survey is such that you might be better off to repeat it with a properly structured survey with easily quantifiable and analysable answers. The main problem that I see there is that is might just p*ss off the patients!

Best wishes


Steve Alker
Unimax Solutions

Hi Omar -

Coding open-ended questions is part art and part science. One way to approach the problem is first by looking for key themes or words that are common among responses. You also need to keep in mind the general idea of your respondents are expressing - for example, price concerns, convenience, product not appropriate to their needs, etc. It all depends on the question they are answering and the level of detail at which you want to code the responses.

I hope this helps - good luck!

Joy