Project SMART

Go to content


Meetings and Conference

SMART: MEETING IN BARCELONA (Barcelona, 29-30 March 2010)

MINUTES OF THE MEETING      < PDF document >

Monday 29th. March:

The morning sessions were devoted to a general discussion how the survey was received and implemented in individual countries. Jacek presented brief quantitative summary. The pilot survey was completed in nine participating countries and the data entered except for 12 last interviews from Germany. The questionnaire was in general well received. However, 13% of the interviewers admitted that it was difficult to convince respondents to participate. This varied from country to country. It seems that respondents from the new EU countries were more difficult to convince to participate in the survey than respondent from the old EU. Most countries applied face to face paper and pencil approach but at least two used CAPI which worked well too.

The data base consisted of 1877 interviews which gives close to 210 interviews per country. Mean and median length of the interview was 35 minutes and only 10% of respondents considered it too long. Taking under consideration that the pilot questionnaire had to be much longer compared to its final version it may be estimated that the average length of the final questionnaire will last not longer than 20 minutes. Vast majority of respondents found the questionnaire clear and well below 10% only considered it fairly unclear or very unclear. Beverage-specific quantity-frequency (BSQF) approach alongside with last occasion (LO) approach were found much easier and better describing respondent's consumption pattern compared to Graduated Frequency (GF). As for individual problems measures RAPS4 and DSM-4 questions were considered much easier compared to AUDIT. Only 6% of respondent found RAPS4 most difficult to respond compared to two other measures (DSM - 15%, AUDIT - 24%).

Representatives of individual countries offered their qualitative feedback and assessments. A number of issues arose for discussion:

  • Some countries had found difficulty persuading people to participate. This may have been because of suspicion of authorities/ use of the research, people feeling 'over-researched' or the subject of the survey (alcohol). Suspicion of research seemed to be a feature in 'new' EU countries.
  • There was quite a difference between countries in length of time to complete the interview - a mean of 23 minutes in Spain and 45 minutes in the Czech Republic. Reasons for the difference were unclear.
  • There were some problems with translation of some sections and concepts. This recurred as an issue later on as well. Countries will look at this carefully and some data may have to be excluded.
  • Concepts such as 'drunkenness' and 'illegal' production and 'unrecorded' consumption were discussed throughout the meeting and clearly presented problems in cross national research. In relation to 'unrecorded consumption', it was mentioned that this is useful in getting an estimate of the accuracy of population statistics/ market movements/ taxation but not so useful for measuring individual drinking patterns. So, we need to be clear about the purpose of this question. These issues will be examined further in the qualitative write up of the data.
  • Researchers described their research procedures and methods, indicating a considerable variation in the way the data had been collected - 'random walks', snowballing, 'networks' of friends and acquaintances etc. Heavy drinkers were also recruited in different ways. However, all countries complied with the guidelines regarding quotas - gender and age groups.
  • It was emphasised in this session and throughout that the country samples could not be regarded as representative and that the data could be used only for examining optimum measurement tools and procedures etc. It was not suitable for a comparative analysis of consumption patterns.
  • Questions of cultural sensitivities were raised e.g. in Hungary 'drinking' means heavy drinking, especially for women, possibly making them reluctant to participate; policy questions were not well received in some cases as in some countries drinking is still seen as a personal issue; etc. There are cultural issues around measuring 'drinking occasions' and 'heavy drinking' e.g. does 6 drinks measure a level of 'risk'? It was suggested that we might look at drinking patterns across Europe rather than within culture - broken down by characteristics such as age/drinking patterns/ other factors.
  • It was reported by several participants that the GF was not liked while he BSQF was generally accepted by most as performing well across Europe.
  • The question of how meaningful it is to use the concept a 'standard drink' was raised as in most countries a concept of 'standard drink' sounds artificial. It was suggested that different levels of measurement might be needed.
  • It was questioned whether 12 months was too long a period to expect good recall (but everyone did not agree on this).

Two afternoon sessions were given over to presentation and discussion of some preliminary quantitative findings from the questionnaires, led by Janusz as regards consumption and three major problems scales. He explained how he had conducted necessary conversions to categorise the data. His presentation and discussion that followed indicated the following:

  • There may still be coding problems and each country needs to clean and check their data.
  • There are big differences between countries in results achieved so far which can partially be distributed to different sampling procedures. Nevertheless, BSQF does give the highest estimate of annual consumption for the whole nine-country sample while LO and GF seem to be inferior with this regard. There are, however, three exceptions namely in Estonia, Finland and Poland where LO offers slightly higher estimates.
  • In consequence, standard deviation of annual consumption is clearly higher in BSQF which may suggest that that method covers the widest spectrum of consumption levels.
  • BSQF shows also the highest dispersion around the median as measured by the range between the first and the third quartiles.
  • Preliminary analysis of variance suggests that BSQF approach is more universal while LO method is more country-specific.
  • Three problem measures applied, namely DSM-4 questions, AUDIT and RAPS have not given consistent results within individual countries.
  • To assess a predictive value of different consumption measures as regards alcohol problems a decision tree analysis was applied controlling for country. Preliminary results suggest that the best prediction is offered by two consumption measures: GF in combination with AUDIT allocating correctly 94% of "normal" drinkers (AUDIT score lower than 8) and 65% hazardous drinkers followed by BSQF: 86% of "normal" drinkers and 75% of hazardous drinkers. A utility of AUDIT in this context has its limitations, however, as AUDIT itself has two items measuring consumption which to some extent implies assessing predictive value of one consumption measure by another consumption measure. If AUDIT is put aside, a predictive value of both GF and BSQF gives decent results with RAPS: 85% "sensitivity" versus 55% "specificity" and 85% versus 52%, respectively. A predictive value of LO for all three problem scales is much weaker and a predictive value of all consumption measures as regards DSM- 4 questions is clearly poorer.
  • ROC curves calculated for all three consumption measures on AUDIT showed very similar curves for GF and BSQF and much "worse" curve as regards sensitivity and specificity of LO. This relation holds for all gender and age groups except for youngest respondents (18-24 years) where all three consumption measures show almost identical ROC curve on AUDIT.
  • Question was asked whether an analysis should be done without the 'extra' heavy drinkers.
  • Could differences in sample size between countries be a factor in the differences found between countries regarding the measures?
  • The usability of DSM in general population samples was raised.
  • The point was also made that we have no external criteria - no 'gold standard' against which to measure and decide which is the best instrument.
  • Next steps in data analyses were suggested including application of item response theory

Last afternoon session discussed preliminary results of sections on individual and community problems. Small corrections were proposed in individual questions which in general measured what they were intended to measure. Factor analysis of attitudes towards alcohol policy showed four factors: alcohol availability control (4 items), advertising control (2 items), drunken driving (two items) and liberal attitude (3 items). Reduction of a number of items was discussed.

Tuesday 30th, March

The morning sessions were devoted to more in-depth examination of the different measures; discussions were held as a whole group and in smaller working groups.

Group reports

Group on consumption measures (Ladislaw Csemy, Esa Ísterberg - rapporteur, Lidia Segura, Janusz Sieroslawski)

We first wanted to look at conclusions that could be made based on the analysis made thus far, and what kind of analysis has to be made in the future. It was noted that similar questions are not necessarily similar in all countries, and that the effects of rotation should be studied. Last occasion questions seem not to be working in Spain which calls for checking the data also in other countries. The question was then raised, what kind of variables could be used in cross-checking. Differences in sampling techniques should be taken into account as well as translation process because of different cultural meanings. Checking the data was seen important because it could save some respondents who otherwise were to be eliminated because their answers were not logically acceptable. Each country should do a consistence control for their own dataset and proposals for deleting some cases and propose some rules how to treat them.

Heavy drinkers and normal drinkers should be analysed separately. Differences of results of heavy drinkers with regard to consumption level according to different methods were felt to be smaller than with regard to normal drinkers. Older population have more regular drinking habits and therefore QF gives for them higher consumption level than last occasion method whereas among younger people the three consumption measurement methods give more similar results. The time of the last occasion should be checked to reveal if it was a time of a special event (New Years Eve; Christmas). For instance in Poland last occasion usually gives the highest consumption estimate, this time the lowest. Holidays have different effects because Czech people for instance are clearly more secular compared to still religious Polish people.

Also algorithms behind has an importance like an assumption as regards the number of days in a year if people have been drinking almost every day. It was also discussed to use data from different studies to control SMART pilot study data.

At the moment the last occasion method seems to be the worst one but after checking data for internal consistence these result may change.

Age should be used more as a controlling variable. The question was also raised if countries could be used more as variable in order to reveal cultural differences.
We were reminded that our basic task in SMART project is to find the best method for all countries. Janusz reminded us that the criteria for finding the best method were dealt in previous meeting (coverage; comparison to recorded consumption figures). In SMAR pilot study the consumption figure is per drinker 15+ (not per capita nor per 15+). The European Health Survey 2013 puts pressure on SMART project to find the best instrument.

Group on problems section (John Foster - rapporteur, Antoni Gual, Ann Hope, Ludwig Kraus)

It was recommended to identify inconsistencies across individual items - with particular reference to ordering of questionnaires. There are a number of items that are repeated- "eg remorse after drinking." Establish what is the "gold standard"/reference point by which to make comparisons.

A question was discussed of forthcoming DSM-V in 2011-2012 probably relating to wording of abuse criteria.

Following questions were discussed as regards AUDIT/DSM/RAPS:

  • Correlate data from first 2 AUDIT items with BSQF, AUDIT 3 with RSOD, AUDIT Dependency subscale with DSM, AUDIT 5,6, 7, 8 with RAPS.
  • Identify hazardous, harmful drinkers, risky drinkers?
  • There is scope for repetition- elimination of items?
  • Compare: Order of severity items- across countries after plotting of curves.

Discussion on section on Adverse Social Consequences (ASC) raised number of questions:

  • Is there an overlap with DSM-IV abuse items?
  • Is there an overlap with AUDIT- 7,8,9,10?- is AUDIT a gold-standard for alcohol-related problems?
  • Does ASC provide additional data - are there gender and age effects that justify its inclusion in addition to AUDIT?

As for the section on heavy drinkers (HD) in your life opinion was expressed whether this relates to alcohol-related social problems? Context is doubtful. Does this fit within the aims of the SMART study? There is likely to be a link to community problems section (COM). Recommend - do not use HD items.

Suggestions for Further Analysis:

  • Item response.
  • Factor analysis using consequence data.
  • Cross country analysis.
  • Separate effects re heavy drinkers v others - i.e consumption levels.
  • Factor Analysis COM data.
  • General elimination of weak items throughout.- insufficiently exact e.g what is a family problem- e.g severity? - time scale, attributable to alcohol?

Group on sections on community problems (COM), alcohol policy (AP) and socio-demographic (Alina Allaste, Allaman Allamani, Kaisa Kaha, Jacek Moskalewicz, Esa Ísterberg, Betsy Thom - rapporteur)

The cultural sensitivity of these questions was raised again and it was suggested that some questions could be added on the positive aspects of drinking. Finnish examples (6 questions) were suggested as a model. These questions could be an option for each country.

It was agreed that the COM questions need to be refined. COM 2/ 4/ 5 are very similar and can be reduced. Factor analysis could be undertaken to see which questions might be eliminated.

Factor analysis would also be useful on the AP questions. Agreement was reached regarding the omission of some items.

Socio-demographic: Apart from adding 'separated' to the 'divorced' category, this was felt to be OK as it stands.

The afternoon session focussed on planning what needed to be done now. The suggestions included:

1. Each country should examine their data for consistency - but we need some criteria for deciding on consistency. Countries should detect any possible errors in their own data set; clean their own data (looking for "wrong cases", "checking for reasons for outliers." Is there a defect in the questionnaire wording- translation?

2. We need to end up with the same data set (co-ordinated by Janusz) for everyone to share.

3. The above should be completed in April.

4. Final versions of all country reports should be submitted by mid-April.

5. During May, more sophisticated analyses will be conducted for the meeting in June at KBS.

6. The KBS meeting will be used to review the situation and suggest the next steps.

7. The final questionnaire and report will need to be completed well before end August (unless a project extension is granted)

8. The final conference will take place at the end of August (unless an extension is granted)

9. Publication of results in a journal special issue. Drafts to be submitted early autumn and hopefully published by end December 2010. Following division of labour was agreed:

  • Elaboration of Kim Bloomfield's paper Methodology of survey research. Ann to collaborate.
  • Review of European surveys. Janusz and John to collaborate.
  • Literature review - Public support for alcohol policy. Jacek, Esa and Thomas to collaborate
  • Methodology paper from this survey. Janusz, Ludwig and Antoni to collaborate
  • Qualitative data/ focus groups - concepts: Zsuzsanna, Airi-Alina, Kaisa and Betsy to collaborate.

Notes by Betsy supplemented by Esa, John and Jacek

Back to content | Back to main menu