METHODS

AIMS AND OBJECTIVES

Recent years have seen a proliferation of research on political parties as one of the key democratic institutions connecting citizens and the state. But most data has been collected in Europe. How do parties compare in nations around the globe? This study addresses this issue, comparing party ideologies, issue positions and rhetoric.

The Global Party Survey (GPS), an international scientific study directed by Pippa Norris, is designed to compare political parties around the world. 

GPS-2019

The first survey in 2019 drew on data gathered from 1,861 party and election experts. It used 21 core items to estimate key ideological values, issue positions, and populist rhetoric for 1,127 parties in 170 countries.

GPS-2023

The second survey is repeating many items from the first study, to monitor stability and change in party positions during recent years, and also adding new items measuring party positions towards democracy and authoritarianism in greater depth. Fieldwork for the second GPS survey is being conducted in spring 2023.

Methods in expert surveys

The GPS builds upon the methods established more than three decades ago in previous cross-national expert surveys designed to identify party ideological and issue positions, including those by Castles and Mair (1984), Huber and Inglehart (1995), Ray (1999), and the series of Chapel Hill Expert Survey of Party Positions (CHES) (Hooghe 2010, Bakker et al. 2012, 2015).

Most previous research using this technique has traditionally focused on North America and Western European democracies and post-industrial societies, although coverage expanded to Eastern Europe (Whitefield et al 2007), and to Latin America. 

By contrast, the Global Party Survey aims to expand the geographic scope of coverage available for comparison, including countries with political parties in all inhabited continents.

Expert surveys supplement alternative approaches to identifying party positions. This includes the long-standing Manifesto Project, extracting data from hand and automatic textual coding of party platforms (Budge 2000). Other methods involve national and cross-national surveys monitoring the attitudes and values of party members and supporters (van Haute and Gauja 2015), party candidates for elected office like the Comparative Candidate Survey, and legislators in national and European parliaments (Bailer 2014). A related body of research has used the techniques of discourse analysis to compare the populist rhetoric used in leadership speeches over time and across countries (Hawkins 2009, Bonikowski and Gidron 2016, Hawkins et al 2019).  All these approaches facilitate triangulation and comparison across datasets provides independent health checks on the reliability and robustness of datasets gathered through these separate studies.

QUESTIONNAIRE DESIGN

The questionnaire used for the Global Party Surveys include a series of core items measured using 0-10 point continuous scales. These are designed to identify the current ideological values, issue positions, and rhetoric of political parties represented in the lower (or single) House of Parliament/Congress. Several items replicate those contained in the Chapel Hill Expert Survey (CHES), to facilitate robustness tests of estimates where the same parties and countries were covered by both studies.

Questions with 0-10 point scales are designed to monitor each political parties’ overall position on four core dimensions: including their values towards the role of the state and markets in the economy, their position towards social liberalism or conservative moral and cultural values, their position towards democratic and authoritarian governance institutions and norms, and their use of populist rhetoric. An example of the typical question design used throughout the study is illustrated below.

The questionnaire is made available through an optional drop-down menu in six major world languages (English, French, German, Spanish, Russian, Mandarin).

GEOGRAPHIC COVERAGE

The global coverage of the survey involves all independent nation-states (defined by membership of the UN) with several important exclusions. The study does not cover dependent territories, like Puerto Rica or Hong Kong. The survey also excluded several countries like Saudi Arabia without de jure or de facto elections for the lower house of the national parliament, or those with constitutional or legal party bans where only independent candidates are allowed to stand for elected office. The study also does not cover micro-states like Tuvalu, defined as those with total populations less than 100,000, in part because of the pragmatic difficulties of identifying sufficient numbers of party and election scholarly experts in smaller societies.  In addition, societies are dropped from the final dataset if they have a poor response rate, making estimates unreliable.

The datasets include party-level and country-level metadata derived from merging selected variables from standard sources, using the latest data available, including from the Varieties of Democracy project (V-dem) and the World Bank Development Indicators. 

PARTY COVERAGE

The list of parties in the survey included up to ten of the largest parliamentary political parties in each country, defined by their share of parliamentary seats in the lower house. In total, GPS-2019 covered 1,127 parties worldwide.

A comprehensive list of parties worldwide is challenging to establish, since the names (and acronyms) of loosely institutionalized parties and party coalitions can shift rapidly over time. There are particular problems in states with legal bans on party organizations and many independent candidates. The list of the English names of parliamentary parties (and their share of seat and vote) were identified from the latest elections for the lower house of the legislature in the year prior to the survey, using the IFES Election Guide. The year of the election data is specified as a variable in the survey dataset.

Wherever possible, the political parties in the survey were given standardized codes using the latest available version of Party Facts to facilitate merging metadata from other national-level, party-level and individual-level datasets.

EXPERTS

Participation in the survey is by personal invitation only. Experts are defined as scholars of parties and elections selected for each country drawing upon the global database established by the Electoral Integrity Project, carefully checked and verified according to several criteria.

Experts are defined as political scientists (or other social scientist in a related discipline) who have demonstrated knowledge of the electoral process and parties in a particular country, such as through publications, membership of a relevant research group, or university employment. This pool has been supplemented in a few smaller countries, like island states in the Caribbean, by several additional scholarly party experts suggested by respondents using the ‘snowball’ technique. Respondents are asked to identify party positions in one country, reflecting their primary area of expertise. The survey included both resident (domestic) and international experts (e.g. a scholar teaching at an American university who specializes in Egyptian or Liberian politics). Two-thirds of respondents in the first survey were born in their country of expertise, while three quarters were a citizen of that country.

As well as the core items, the surveys also include several questions about the nationality and citizenship, demographics (gender, age), party preferences and the self-reported L-R ideology of the experts, as well as their reported familiarity with each of the parties and the difficulty they experienced in completing the survey. One quarter of the experts in the first survey were female, reflecting gender disparities in the discipline. The mean position of experts on the self-reported Left-Right ideological scale in GPS-2019 was 4.75, close to the mid-point. Overall on the difficulty or ease of completion scale, most respondents reported positively that it was easy (7.82). Respondents are generally more familiar with the larger parties in their country of expertise, suggesting that care should be used when analyzing the estimates for smaller and newer parties. This data allows users to analyze whether these characteristics influence estimates, such as whether the ideology of experts influenced their assessments of party positions (Curini 2009).

FIELDWORK

Personalized survey invitations are distributed by email to experts, and responses collected online, through the Qualtrics platform. Fieldwork is conducted for one month, with an initial personal invitation to individual experts followed by two reminders. Fieldwork for the first survey ran from 19 November to 20 December 2019, immediately prior to the Covid outbreak.

RESPONSE RATE

In total 1,891 experts participated in GPS-2019, representing an overall response rate of 23%. On average, each country included replies from around a dozen experts. Fewer replies were received from smaller developing societies, however, in states governed by autocratic regimes restricting freedom of expression, and in regions like MENA where few university scholars study parties and elections. For example, while around 19 experts responded on average in liberal democracies, on average around 5 respondents responded in most closed autocracies.

The number of experts per country is included as a variable in the dataset (Expert#) so that users may choose to adopt a minimum threshold, deciding to exclude cases which fall below a specific number of country responses. A measure of expert familiarity with each political party in each country was also included (V1), allowing users to filter or weight the estimates based on this data.

ROBUSTNESS TESTS

Robustness tests help to check the external validity of the GPS data. Previous research has used this process to check the reliability of the CHES estimates when compared with several independent data sources, such as those from party manifestos, surveys of MPs and other expert surveys (Bakker et al., 2014; Hooghe et al., 2010; Marks et al., 2007; Netjes and Binnema, 2007; Steenbergen and Marks, 2007; Whitefield et al., 2007).  

This method can also be used to evaluate the reliability and validity of key items in the GPS-2019 survey, compared with the previous ParGov and CHES estimates.

In total, 192 parties in the GPS-2019 study could be compared with ParGov data. The ParlGov project estimated the position of political parties on a 10-point left-right scale by combining data from previous expert surveys conducted by Castles/Mair 1983, Huber/Inglehart 1995, Benoit/Laver 2006, and CHES 2010. The GPS-2019 estimates of the economic position of parties on the left or right (V4) proved to be strongly correlated with ParlGov left-right estimates (R=0.739***), as well as with ParlGov’s estimate of whether parties favored regulating the economy, derived from Benoit/Laver 2006 and CHES 2010 (R=0.820 ***).

Similarly, ParlGov generated a 0--10 mean value scale for the position of parties in the 'libertarian/authoritarian' dimension, with data combined from Benoit/Laver 2006 and CHES 2010. This ParlGov measure was strongly related to the GPS estimate of party support for liberal or conservative social values (R=.828 ***).

Some differences among expert estimates are only to be expected, given different time-periods, question wordings, and research designs. Parties are far from static in their issue positions and ideological values, given changes in leaders, policy agendas, and patterns of competition. Nevertheless, triangulation using replication tests from previous studies generally confirms the external validity and robustness of the GPS-2019 estimates on several key ideological indicators. 

The technical note and codebook describes the results of the robustness tests.

ADMINISTRATION

The survey is directed by Pippa Norris at Harvard University. The first survey was funded by the Australian Research Council Laureate award as part of the Electoral Integrity Project. The EIP expert database has been compiled over the years by several survey managers, including Max Grömping, Ferran Martinez i Coma, Alessandro Nai, Richard Frank, and Thomas Wynter. The GPS survey was administered by Laura Welty at the University of Sydney.

The second survey has been assisted by the research assistants, Jorge Ruiz and Luke Willams, both studying at Harvard University, who updated the expert database and expanded the list of identified expert.

The original research design was developed in conjunction with colleagues at the Chapel Hill Expert Survey (CHES), Liesbet Hooghe, Gary Marks and Ryan Bakker, the team conducting a similar survey in Europe and Latin America.

Several other scholars of parties and elections were consulted about the early draft survey design and they generously offered invaluable suggestions, for which many thanks are due, including Mark Franklin, David Farrell, Tim Bale, and Susan Scarrow.  Finally, the research could not have been completed without the time and effort which all the experts invested in completing the survey - for which, many thanks.  

DATASETS

The GPS-2019 dataset is available for download from:

https://dataverse.harvard.edu/dataverse/GlobalPartySurvey

Datasets are available in Excel, SPSS and Stata formats at the levels of experts (GPS-2019-EXPERTS), and parties (GPS-2019-PARTIES).

Please contact us with any corrections needed or suggestions to improve the dataset.

30 Jan 2020

References

Bailer, S.  2014. ‘Interviews and surveys in legislative research.’ In S. Martin, T. Saalfeld, and K. Strom. (eds) The Oxford Handbook of Legislative Studies. Oxford: Oxford University Press.

Bakker, Ryan et al. (2012). ‘Measuring party positions in Europe: The Chapel Hill expert survey trend file, 1999–2010.’ Party Politics 21(1).

Benoit, Kenneth and Michael Laver (2006) Party Policy in Modern Democracies. London: Routledge.

Bonikowski, Bart and Noam Gidron. 2016. ‘The populist style in American politics.’ Social Forces 94(4) 1593–1621.

Budge, Ian (2000) ‘Expert Judgments of Party Policy Positions: Uses and Limitations in Political Research’, European Journal of Political Research 37(1): 103-13.

Castles, Frank and Peter Mair (1984) ‘Left-Right Political Scales: Some Expert Judgements’, European Journal of Political Research 12: 73-88.

Curini, Luigi. (2009) ‘Experts’ Political Preferences and Their Impact on Ideological Bias: An Unfolding Analysis based on a Benoit-Laver Expert Survey.’ Party Politics 16(3) https://doi.org/10.1177/1354068809341051

Hawkins, Kirk A., Rosario Aguilar, Erin Jenne, Bojana Kocijan, Cristóbal Rovira Kaltwasser, Bruno Castanho Silva. (2019). Global Populism Database: Populism Dataset for Leaders 1.0. Team Populus. https://populism.byu.edu/Pages/Data

Hawkins, Kirk. 2009. ‘Is Chávez populist? Measuring populist discourse in comparative perspective’ Comparative Political Studies, 42(8): 1040-1067.

Hooghe, Liesbet, Gary Marks, Carole J. Wilson. 2002. ‘Does Left/Right Structure Party Positions on European Integration? Comparative Political Studies 35(8).

Hooghe, Liesbet, Ryan Bakker and A. Brigevich. (2010) ‘Reliability and validity of measuring party positions: The Chapel Hill Expert Surveys of 2002 and 2006.’ European Journal of Political Research 49(5): 687–703.

Huber, John and Ronald Inglehart (1995) ‘Expert Interpretations of Party Space and Party Locations in 42 Societies.’ Party Politics 1: 73-111.

Keman, Hans (2007) ‘Experts and Manifestos: Different Sources - Same Results for Comparative Research?’, Electoral Studies 26: 76-89.

Mair, Peter (2001) ‘Searching for the Positions of Political Actors: A Review of Approaches and a Critical Evaluation of Expert Surveys.’ In Michael Laver (ed) Estimating the Policy Position of Political Actors, pp. 10-30. New York: Routledge.

Marks, Gary, Hooghe, Liesbet, Steenbergen, Marco R. and Bakker, Ryan. (2007). ‘Cross-validating data on party positioning on European integration.’ Electoral Studies 26 (1): 23-38.

Netjes, C, Binnema, HA. (2007) ‘The salience of the European integration issue: Three data sources compared.’ Electoral Studies 26(1): 39–49.

Ray, Leonard. (1999). ‘Measuring party orientations towards European integration: Results from an expert survey.’ European Journal of Political Research, 36, 283-306.

Steenbergen, Marco R. and Gary Marks (2007) ‘Evaluating Expert Judgements’, European Journal of Political Research 46(3): 347-66.

van Haute, Emily and Anika Gauja. (2015) Party Members and Activists. Abingdon: Routledge.

Whitefield, Stephen, Vachudova, Milada Anna, Steenbergen, Marco R., Rohrschneider, Robert, Marks, Gary, Loveless, Matthew P. and Hooghe, Liesbet. (2007). ‘Do expert surveys produce consistent estimates of party stances on European integration? Comparing expert surveys in the difficult case of Central and Eastern Europe.’ Electoral Studies 26(1): 50-61.