Hot off the Presses: New Critical Reflections on Measuring Media and Development

Just published: a welcome new contribution to the cause of measuring media and its impact on development.  The volume of scholarly papers, Measuring Press Freedom and Media Contributions to Development: Evaluating the Evaluators, was edited by Monroe Price, Susan Abbott and Libby Morgan. The book is described as “bringing together a variety of viewpoints and perspectives on evaluating media assistance, Measures of Press Freedom and Media Contributions to Development offers a critical reflection on the theories and tools of measurements that are used by the academic, donor, and civil society communities.”  The book promises to significantly advance current thinking on how best to measure media and its impacts.

For a shorter, more policy-focused take on the volume, look at John Burgess’ report for The Center for International Media Assistance based on this book called Evaluating the Evaluators: Media Freedom Indexes and What they Measure.

Tara Susman-Peña is Director of Research, The Media Map Project

 

 

The limits of press freedom: Mali

The Media Map Project was in Mali in February to research the impact of donor-funded media development interventions over the last 20 years.  While we’re still preparing our findings on that topic, I wanted to share some observations.  Some striking characteristics of Mali’s media landscape are directly related to the discussions we have been having on this blog about the advantages and limits of using Freedom House’s Freedom of the Press Index as the go-to dataset to stand in for media development.  Freedom House gives Mali a “Free” rating.   Our field research supports this assessment.  Our desk research also shows that good empirical evidence already exists that supports a strong connection between the media and development.  In a soon-to-be released report outlining how quantitative data has been used to measure the health of a country’s media sector in the context of economic development, Sanjukta Roy writes, “The academic literature,  through theoretical models and empirical testing, has validated the role of the media in facilitating good governance and favorable developmental outcomes.”  The index that these studies have overwhelmingly used to measure the media sector?  Freedom House’s Freedom of the Press Index.

But does that mean that we can assume that a country with a Free Press rating will have similarly high ratings in governance and development indexes?

Unfortunately, no.  But given the way that the academic literature has emphasized the Free Press measurement, it’s hard not to assume that a Free Press rating also implies a healthy media sector and positive development milestones when you narrow the focus to the country level.

Sadly, in Mali, that is not at all the case.  Mali has been free from colonialism for 50 years and is celebrating its 20th year of democracy.  Mali is also one of the ten least developed countries in the world, by various different measures of development.  Mali has a notably higher mortality rate, lower level of education, and worse environmental challenges than sub-Saharan Africa as a whole, and indeed, low income countries when taken as a group.  The government of Mali is democratic, but not transparent about its activities.

What about the media in Mali?   Yes,  Mali’s press is free.  But our trip to Mali vividly illuminated that the rating “Free,” accurate though it may be, cannot be taken to represent the health of Mali’s media sector.  What we saw in Mali was that the media system is characterized by disorganization, hyperbolic and sensationalized reporting, and a lack of professionalism overall.  Not a single journalism school exists.  A harsh libel law is still on the books and is enforced from time to time.  Most media, whether private or community, cannot sustain itself as a business, and so brown envelope journalism is alive and well.  Almost 75% of citizens are illiterate, limiting the types of media most people can access.  Many Malians also lack the ability to discern the difference between good and poor quality information.  No comparable, good quality data on the media market or on media use is collected consistently.   Thus, it is difficult to assess the media across the country and over time from either a business perspective or from the citizens’ perspective.

It’s not all bad news. Recently the Minister of Health had to step down because the media exposed the misappropriation of international grant funds going on in his office.  Community radio is alive and well, with over 250 stations spread throughout much of the country.  Mali has been connected to the Internet since 1996 (though to put that in context, the number of internet users is still far lower than the average for low-scoring sub-Saharan Africa), and mobile penetration is on the rise.

Media has received a lot of support from donors over the last 20 years, but seems to be less of a priority for donors now that the Malian democracy is pretty effective.  But clearly the media system still has a long way to go.

So what to do?  What would be the best approach to leverage Mali’s Free Press into a developed media system?

Tara Susman-Peña is Director of Research of the Media Map Project.

Measuring Media Development pt. 2

The trials and tribulations of finding reliable proxies for media development continues! As I discussed in my last post, I’ve been searching for some way to quantify media development, by combing through data that other, larger organizations have already collected. I’ve been using the Media Sustainability Index’s 5 criteria as a template. The unfortunate problem with the MSI is that it only covers a limited number of countries, and for a limited number of years. I discussed in my previous post, this makes conducting statistical analysis difficult.

I thought it would be interesting to share some of the information that I’ve been gathering. I’m very receptive to your comments or ideas, and it might be useful for you to know exactly how hard we are working to get good data! I drew from a variety of datasets, most notably the World Bank Development Indicators, the Global Competitive Survey, the Global Integrity Report, and the Institutional Profiles Database. So, without further ado, I present some tentative proxies that may get at media development.

MSI Criteria 1: Legal and social norms protect and promote free speech and access to public information.

This may be one of the easier criteria to satisfy, as several organizations are interested in measuring legal and social norms. Reliable measurement is always an issue when trying measure something as amorphous as ‘norms’; however, survey data and expert opinion is really the only feasible way to get at some of these issues. Most notably, the Freedom House Freedom of the Press index covers most countries for several years, which clearly gets right to the heart of free speech. Additionally, their Freedom in the World data analyzes civil and political liberties, which are tied to social norms promoting freedom of speech.

The Global Competitiveness Report measures indicators that may serve as a proxy: intellectual property rights, antimonopoly policy, burden of regulation, and auditing and reporting standards. Some of these may also fit into MSI Criteria 4 and 5, but having variables overlap doesn’t it lend itself to rigorous analysis, so I keep them here.

The Institutional Profiles Database also covers some of the MSI 1. They have measures for freedom of association, freedom of movement and peoples, and emulation of neighboring countries.

The Global Integrity Report quantifies Rule of Law.

MSI Criteria 2: Journalism meets professional standards of quality.

Unfortunately, I have yet to come across anything that I think even begins to address this criteria. Journalist salaries may be one way of looking at this issue, though this data can be difficult to obtain as well. I have some leads on websites relevant to this area, so stay tuned. Ultimately this area may be the most difficult of the 5 to approximate using other sources.

MSI Criteria 3: Multiple news sources provide citizens with reliable, objective news.

Global Integrity Report has some interesting data here. They ask about ‘credible media sources’ and ‘public access to information.’

The World Bank Development Indicators look at how many newspapers there are per 1000 people, which at least gives us an idea that there are newspapers. It also looks at internet, radio and television use. It sadly does not tell us about the quality, nor even the number of news sources.

MSI Criteria 4: Independent media are well-managed businesses, allowing editorial independence.

There’s a lot of information about businesses in general, but less about media as a business. So most of the proxies I found deal with business as a whole, and so by extension, good business environment may also mean good media business environment, although that may be stretching a little bit. Perhaps a good way to put fears to rest is to see if there is a correlation between business environment and freedom of speech.

The Global Competitiveness Report asks about ethical behavior of firms, and the procedure to start a business. Additionally, Institutional Profiles Database also asks about the ease of starting a business, and the World Bank Development Indicators has data on the number of new businesses registered and the ease of starting a business. The Global Integrity Report also examines business and regulation.

MSI Criteria 5: Supporting institutions function in the professional interests of independent media

This last criteria can be taken extremely broadly. I included the Global Competitiveness Report’s ‘Electricity supply’ and the World Bank’s ‘Energy Use’ into this category, because I felt that stable electricity is an important asset for media development. As most communication is done using radio waves or televisions (and the internet!), a steady source of electricity may be an important variable. What may be interesting to look at is how electricity supplies actually affect media development, because there may be no causal relation at all.

The Global Competitiveness Report also includes a ‘Technological readiness’ measure, and the World Bank has a measure for Science and Tech Research and Development spending. Again, this may be stretching the idea a bit too far, but there will be time to refine the variables as the project moves along.

Another important institution that supports media development is education. The World Bank has data on literacy rates per country, which may be a useful proxy for the quality of education. Likewise, the Global Competitiveness Survey measures a ‘Quality of Education.’ One would hope that high levels of education help lead to professional standards, quality media reporting AND an interested audience.

More work should be done on finding other supporting institutions. Rule of Law and fair judiciaries may also be proxies to include in this section. As the project develops, some indicators may switch around, and we may throw some out altogether. It’s about the process, right?

Each of these datasets have their own unique benefits and drawbacks. Like the MSI, most of them do not span each country, and the country-years are generally low, meaning that the datasets only cover a few years. However, by compiling this list, we’re getting a little bit closer to having a wide range of data that help us to better measure media development. Unfortuantely, large scale, reliable datasets spanning 30 years are hard to find (and if they exist, they may not even cover the necessary areas!), so trying to piece together something useful is a challenging endeavor.

Kim Johnson is a Master’s Student at the School of International and Public Affairs, Columbia University.

Measuring Media Development

What is “media development”, exactly? While press freedoms are clearly an important part of media development, there’s a variety of other factors that we consider equally important to media development: Are there several media sources, or only a state owned television station? Are journalists professional? Do they receive fair wages for their work? Is there infrastructure in place to support television or radio stations? Are citizens able to freely access information? Are journalists independent from the state? Although these factors aren’t exhaustive, we can see that there’s a lot to consider in regards to media development.

Like democracy indicators, there’s a variety of different indices available that deal with media. A few of the more widely known indices are Freedom House’s Freedom of the Press, IREX’s Media Sustainability Index and Reporter’s Without Borders Press Freedom Index. This list is not exhaustive; there are plenty of other organizations (such as World Bank or the UN) out there that also try to provide quantifiable measures.

And although these indices are wonderful, providing detailed information about a variety of countries, not all of them address the issues we are concerned with when we talk about “media development.” For instance, Freedom House’s Freedom of the Press focuses on the treatment of journalists, examining legal, financial and political environments. Unfortunately, it doesn’t quite get at the full picture of “media development,” because it ignores the key issues of professional standards of journalism, infrastructure, and citizens’ access to media. This is really regrettable, given Freedom House’s index is extensive, covering 196 countries, starting in 1980. Because of it is so extensive in terms of time and countries covered, Freedom House has been the go-to index for many researchers, but freedom of the press isn’t the whole story of media development.  This post isn’t meant to critique Freedom House; rather, it is to highlight some of the difficulties the Media Map team encounters when it tries to find measures of “media development.”

Even when an index does seem to measure our idea of media development, other issues arise. For example, IREX’s Media Sustainability Index (MSI) covers five major objectives:

1. Legal and social norms protect and promote free speech and access to public information.
2. Journalism meets professional standards of quality.
3. Multiple news sources provide citizens with reliable, objective news.
4. Independent media are well-managed businesses, allowing editorial independence.
5. Supporting institutions function in the professional interests of independent media.

The objectives seem to get a the heart of “media development,” right? They aren’t focused solely on press freedoms, but look at broader institutional supports, and address issues of professionalism. Unfortunately, this index doesn’t cover as many countries as the Freedom House index, and the project started in 2005, making statistical analysis more difficult.

As a research assistant for the Media Map project, I tried to think of a way to get around these issues. So far our main issues are that most indices don’t exactly measure media development, and the one that seems to doesn’t have enough data. Looking at the MSI objectives, I wondered if we could use data from other organizations as proxies for the MSI objectives. Surely there exists data on supporting institutions in more countries than the ones that the MSI provides; likewise, information about legal and social norms.

The others on the Media Map team thought this sounded like a good idea, so I was given the task of combing through a variety of indices, as well as data provided by World Bank and the UN to see if I could find this information available. So far, I’ve found some indicators that may serve well as proxies from the Global Competitive Survey, the Global Integrity Survey, and others. The biggest challenge has been finding data for objective 2, the professional standards. There hasn’t been much work done on professional standards, and we may have to get creative with filling that gap in the future. But hopefully, once we’ve compiled the indicators, we’ll have a more comprehensive dataset that focuses more specifically on media development, spanning a much larger number of countries and years than the current MSI index.

Kim Johnson is a Master’s Student at the School of International and Public Affairs, Columbia University.

Its Just Not About “Democracy”

A democracy is undoubtedly the best form of government. It is considered to be the primary precondition for economic development. There are many different ways in which the condition of democracy is quantified. The key indicators in this regard are:

1) Polity IV Project: This “examines concomitant qualities of democratic and autocratic authority in governing institutions, rather than discreet and mutually exclusive forms of governance”.
2) Vanhanen’s Democratization Index: The VDI is based on two dimensions, public contestation and the right to participate, which are named as competition and participation, respectively.
3) EIU Democracy Index: The index is based on five categories namely, electoral process and pluralism; civil liberties; the functioning of government; political participation; and political culture.

The above list in indicative. There are many more indicators that quantify the extent of democracy in countries in various different ways.

Interestingly however, no clear consensus exist for the causal connection between economic development and democracy in the academic literature. Further, in dire issues like that of terrorism, democracies are deemed to be the most vulnerable. This brings us to the very important question that what are then the required factors that would make a democracy perform to its full potential ? A democracy can only perform to its full potential when empowered by other complementary institutions. A key institution in this regard is Media. A strong, independent and vibrant media sector would help reduce information asymmetry in a society and elucidate the flaws in the same. It is only when the people of a country be aware of the holes in a system and demand remedy would a government feel the pressure for accountability. And it is this constant process of checks and balances that would ensure a country’s journey from being a weak and “flawed” democracy to a strong and “functional” one.

This can be reiterated in simple terms as propounded by Persson and Tabellini (2000):

Free Press —> Voters’ State of Knowledge —> Democracy —> Selection of Political Parties —> Government Responsiveness

Thus to ensure economic development, a democracy and a strong media sector need to function in unison, empowering each other in the way. Any one of them without the other is not the solution.

Hence, it is time that while the extent of democracy has been scrutinized quantitatively in multiple ways, the effectiveness and performance of the media sector of a country be also evaluated in various different ways. We have been relying overtly on Freedom of the Press for a long time. It is time we have other ways of quantifying the strength of the media sector – in multiple different ways and in a cross-country framework.

Sanjukta Roy is the Data Analyst for The Media Map Project.

The Intellectual History of Media Development

The Media Map project is excited to share our literature review focusing on the study of media development as a field.  The report explores the origins of media development research, outlines existing empirical measurements of the impact of media development projects, and examines relevant theories about the relationship between media modernization and societal progress.  In reviewing the literature, it lays the groundwork for The Media Map Project, a study on the impact of media development worldwide.

Literature Review – The Media Map Project

Annotated Bibliography – The Media Map Project

Tara Susman-Peña is the Director of Research of The Media Map Project.

Media and development in Sub-Saharan Africa

Russell Southwood of Balancing Act, experts in ICTs and media in Africa, shares this video blog on the role of the media and key challenges to understanding it in sub-Saharan Africa.

Tara Susman-Peña is the Director of Research of The Media Map Project.

What is convergence, anyway?

Convergence is where different forms of media interact with each other.  The continual evolution of these interactions is a great opportunity for creativity.   For the Media Map Project, convergence has meant finding a number of excellent partners to collaborate with.  One partner, the Centre of Governance & Human Rights at the University of Cambridge, is collaborating with FrontlineSMS to explore the future of community radio and its convergence with mobile.  Read more about their work here.

Tara Susman-Peña is the Director of Research of The Media Map Project.

Will media get a seat at the development table?

This week the Media Map Project team is in Brussels, where we organized a special event for the World Bank Institute’s Strengthening Responsible Business and Governance in Africa program.  We had some interesting discussions leading up to the event on the Bank’s online platform, where participants were quite passionate about the importance of the media in these areas.  And the importance of the role of the media in driving responsible business was a theme that ran throughout several of the events.  It still remains to be seen, though, whether the Bank will take media seriously, both as a development driver, and as a sector needing development attention in many countries.  Without functioning well as a business, the media cannot fulfill its potential to drive development.  There needs to be a lot more serious and concerted thought, at a strategic level and not as an afterthought, about what the Bank can do to improve the development of media, so that it is sustainable as a business and can perform the various roles of

• watchdog of government, industry, civil society, and indeed the media itself
• driver of development
• space for the public to get vital information and communicate their needs and desires to decision-makers

The special session on the role of media came up with these recommendations to support media’s role in responsible business and governance, and to support the sustainability of media as a business.

• Plan an event together with World Bank private sector and governance stakeholders, and the African Media Leaders’ Forum to bridge the gap between governance and business initiatives and media leaders’ concerns
• Construct a multi-stakeholder platform for discussion and strategy on media, governance, and business, to include representatives from governments, private sector, the media, and civil society
• Create opportunities for South-South and North-South learning on this topic
• Plan regional forums in Africa to address these issues and contextualize them according to the specifics of each African region and country
• Identify gaps in data that measures media freedom, strength, quality, reach, and independence (connect to organizations and initiatives already working on this, like PAMRO, Audiencescapes)
• Create independent institutions to use consistent methods to measure the audience of each type of media and advertising expenditure for all countries in Africa. This data is the building block for independent media, without consistent, reliable data that tracks these issues over time, investors will not enter into the market.

We look forward to progress in this area!

Tara Susman-Peña is the Director of Research of The Media Map Project.

Free Press and Investor Confidence – Fact or Coincidence?

During the ongoing analysis of the condition of countries with or without a free press, a very interesting and encouraging fact came to the forefront. While taking a close look at Sub-Saharan Africa (SSA), we found that between 2002 and 2008, the SSA countries with a Free Press (a classification based on Freedom House’s yardstick) have been successful in attracting a sharply increasing inflow of FDI. Infact in these countries the gap between the share of foreign aid to GDP and the share of FDI inflows to GDP seem to be closing down – an indication of investors gaining confidence in these economies.

This is in sharp contrast to the SSA countries with a Not Free Press where the share of FDI inflows to GDP has not seen much change over time and the gap between aid and FDI is actually widening (implying a marginal increase in aid dependence).

This observation is worth a discussion – is free press responsible for the extent of FDI inflow in to economies or is it just a co-existing factor?

FDI vs. Aid

Sanjukta Roy is the Data Analyst for The Media Map Project.