City rankings – information and distortions

Image: João Nitsche
Whatsapp
Facebook
Twitter
Instagram
Telegram

By GREINER COSTA & LUÍS FERNANDO VITAGLIANO*

Rankings produce information that official national statistical institutions do not and should not do: prioritize development and quality of life in cities

In recent months, some news sites have published city evaluation indexes produced by private entities. The titles varied in general with expressions such as: “Sustainable cities”, “Smart cities”, “Cities with better quality of life”, “Most tree-lined cities in Brazil and the world”, “Excellent cities”…

These rankings produce information that official national statistical institutions do not and should not do: rank development and quality of life in cities through evaluation and grades between “best” and “worst”.

Using or not official indicators to construct indices, there will always be a 'bias'. Especially because, intentionality aside, ranking indicators and weighting values ​​through weights and grades generates relevance to one or another element of the calculation. In official institutions there is an effort to eliminate bias. In institutions that voluntarily construct their indices, bias may be welcome as an argument reinforcer.

Election years like the current one are opportunities to give visibility to the strategy of valuing this measurable research. This year, as could not be otherwise, at least two new indices emerged, which timed their release with the proximity of the start of the 2024 electoral period.

Apparently, the production of evaluations of municipal management or quality of life in cities appears as a noble subject on journalistic portals. In essence, the production of indices has become a “market” filled with private entrepreneurs offering prizes, selling advice, etc.

The pre-election period concentrates potential customers eager for news in their favor or against their opponents. It is a thorny terrain where each management team, especially on the eve of an election, can hire “advisories” or can choose an index that favors them to call their own. For the general public, as can be seen in the comments to the articles on the websites, the predominant feeling is often one of strangeness with the results for their city.

The attribution of grades and rankings in which small municipalities, with less than 10 thousand inhabitants, are placed side by side with cities with more than 500 thousand or compared with capitals and metropolises with millions of inhabitants may sound fanciful. Another point that raises doubts is the exercise of comparing municipalities in metropolitan regions, troubled cities, with others of the same population size, but which are far from large centers and are in fact regional hubs. Likewise, it may seem like an oversight, added to the bias inherent in indices, to make comparisons between cities with very different socioeconomic profiles and different per capita incomes.

Out of caution, we chose to create a brief panel of considerations here, observing the recent results of six of these studies for five cities in the state of São Paulo, with populations between 200 and 300 thousand inhabitants (IBGE 2022). Data relating to GDP per capita or territorial extension of each municipality can always vary greatly in each case. The two cuts adopted here, restricting to one state and choosing an average population range, are sufficient to verify discrepancies that actually exist in the published results. Due to the variety of objectives, criteria, methodologies and databases potentially used, it is not appropriate to question results on a case-by-case basis, but rather the general idea of ​​“ranking” between cities.

To select the cities, we found that the state of São Paulo has 16 municipalities with a population between 200 and 300 thousand inhabitants. We defined cities in this population range, one close to the capital, but not in the RMSP, one in the metropolitan region of Campinas, one on the coast and two from different regions in the interior. Others could certainly be defined. In the general picture, the choice does not change the point we seek to analyze: the results of the indices are very diverse and in general are not useful for much other than spreading dubious information to the unsuspecting and to marketers during electoral periods.

Tabela 01 – selected cities

  Guaruja Indaiatuba jacarei Marilia San Carlos
Population 287.634 255.748 240.275 237.627

254.857

GDP per capita   Brazil Position: R$ 36.751,08 1.698º /5570 R$ 89.869,47 263º /5570 R$ 67.472,37 510º /5570 R$ 41.355,48 1.421º /5570 R$ 55.894,44 790º /5570
GDP of the City Position SP:   46th /645   15th /645   28° /645   48th /645   35th /645
Source: IBGE (Own preparation)

The studies analyzed are:

Municipal Management Effectiveness Index (IEG-M). Created in 2015 by the São Paulo State Court of Auditors to measure the efficiency of São Paulo City Halls.[I]

Sustainable Cities Development Index – Brazil (IDSC-BR), from the Sustainable Cities Institute (ICS), in the Sustainable Cities Program (PCS), with support from the Brazilian Center for Analysis and Planning (CEBRAP).[ii]

Connected SMART CITIES Ranking – the smartest cities in Brazil (September/2023).[iii]

IPS BRASIL 2024 – The Social Progress Index is prepared by PS Brasil, in collaboration with other entities in the Amazon and the Social Progress Imperative.[iv]

Best Cities for Doing Business – URBAN SYSTEMS – The study is produced annually for EXAME Magazine. The 2023 edition is the 10th held. [v]

INVESTE SP / GOVERNO DE SP – Entrepreneurial Cities Index (ICE). [vi]

In Table 2, below, a comparative table of the indices attributed to each of the cities in Table 1 is presented.

Tabela 02 – Reviews overview

 

Guaruja

Indaiatuba

jacarei

Marilia

San Carlos

IEG-M TCESP (2023)

C+

B

C+

C+

C

IDSC-BR (2023) Level of development:

50,98 /100 1.305º /5570 Medium

61,30 /100 14º /5570 High

52,63 /100 921º /5570 Medium

52,14 /100 1021º /5570 Medium

57,78 /100 198º /5570 Medium

Smart Cities (2023) – General Position (Brazil)

29,380 72st

31,311 37st

29,762 63st

Not among the 100 listed

Not among the 100 listed

IPS – Brazil (2024)

61,12 /100 1.549º /5570

70,47 /100 6º /5570

64,32 /100 566º /5570

64,61 /100 511º /5570

70,96 /100 3º /5570

Urban Systems, position by Segments (Br-2023) – Commerce: – Education: – Agriculture:

Not mentioned among the 100 Not mentioned Not mentioned

    29th Not mentioned 72nd

71th Not mentioned 12nd

Not mentioned 74th 97th

51th Not mentioned 85nd

INVESTE SP index, (Brazil position)

77th /101

Not on the 101 list

Not on the 101 list

Not on the 101 list

Not on the 101 list

Source: Self elaboration; various sources discussed in this article present in the citation.

Table 2 shows differences between evaluations and scores for cities. Strictly speaking, a large dispersion of scores using consistent methodologies, which come from the same databases, is not expected. One of the important procedures in the construction of statistical research methodology is the observation of discrepancies between indicators. Some points of attention:

According to the IEG-M TCESP Study (2023), all would be in a very close range of evaluation with grades “C”, “C+”, with only Indaiatuba slightly above, presenting a “B” rating;

The worst score in the TCE's IEG-M is that of São Carlos, the best is that of Indaiatuba;

Only Indaiatuba has a level of development considered high by IDSC-BR;

Indaiatuba is the best placed by the general Smart Cities index, ranking 37th among cities in Brazil; in this study Marília does not appear in any of the clippings;

Indaiatuba and São Carlos are in position 72nd and 46th in the Education section of the Smart Cities index, and do not appear among the 100 listed in the Education segment of Urban Systems;

It is strange to see Guarujá, with all the obvious problems with crime reported extensively in recent years, appear in 8th position in the Smart Cities ranking, including security;

São Carlos is ranked 198th in the IDSC-BR, it is not listed in the general Smart Cities index nor among the best cities in the Urbanism, Environment and Technology and Innovation sections of the same index; and it does not appear in the Education segment of Urban Systems nor among the 101 cities listed in INVESTE SP, but strangely it appears in 3rd place nationally in the IPS-BR index;

Only Guarujá is listed by the INVESTE SP index.

It is surprising to see Jacareí ahead of Marília and São Carlos in the Agricultural, Urban Systems segment.

Possible interpretations

It is necessary to emphasize that this is not about disqualifying city assessment work, much less the institutes and professionals who carried it out. In all the studies selected here there is concern with the transparency of the sources and the methodology, databases consulted and procedures for weighting scores according to the focus of interest of each index are reported.

What can be verified, even with this small sample, is that the bias generated by general grades can confuse more than it explains. This divergence shows that it is not the best way for these studies to look for headlines in newspapers and news sites highlighting the ranking. Who would be the smartest city in Brazil or the safest or the best to live in? The question is wrong and leads to distortion. This means of attracting attention brings with it greater scope for doubts, questions, ironies, affecting the credibility of the study due to results that generate a feeling of strangeness in people.

An alternative adopted by TCE SP for the IEG-M is the definition of assessment bands and the cities that would be in each of them. This takes away the ranking. In this way, studies could point out which cities are each year in a situation of sustainable development and/or quality of life with an A grade (the best, high level); others would be at level B, others in the C or D or E assessment range, for example.

This simple change in the scoring and evaluation standard would make unnecessary comparisons unfeasible, which occurs when a city appears as the best in Brazil in some index and for another study it is in position 200 or 900... Noise and energy spent, unnecessary manipulation and controversy.

Finally, a brief observation about the class profile in these evaluations is worth making. Certainly, medium and large cities in the interior of São Paulo can always be on lists of cities well evaluated by criteria such as development, environmental, access to services, education, health or technology.

But it is clear: families with higher incomes or those who live in areas with better urban structures will certainly have a different quality of life or perception of the environmental sustainability of their city compared to those in the same cities who live in peripheral or sub-housing areas. There is a breakdown of income and social class, access to services, etc. that generic and general assessments do not cover or, in the end, the dance of notes hides.

*Greiner Costa is an engineer, specialist in planning and public management.

*Luis Fernando Vitagliano political scientist and university professor.

Notes


[I] Focusing on infrastructure and processes, it evaluates public policies in seven sectors of administration: health, planning, education, fiscal management, citizen protection, environment and information technology governance. In this index, the cities with the best evaluation have a Grade A, the worst will have a C. See using this link.

[ii] The Sustainable Cities Platform brings together a base of 260 indicators associated with the areas of public administration activity and a bank of good practices with exemplary cases of national and international public policies, as references to inspire municipalities. To see using this link.

[iii] The published Ranking is made up of 74 indicators in 11 thematic axes. The study considers the “Concept of Connectivity” such as the relationship between the different sectors analyzed. The concept of “Smart Cities” points out that there will only be development when a city’s development agents understand the connectivity between all sectors; Investments in sanitation are linked not only to environmental gains, but over time they may also reduce basic health care and generate impacts on governance and the local economy. The 2023 edition of this Ranking collected data and information from Brazilian municipalities with more than 50 thousand inhabitants (CENSO IBGE 2022), totaling 656 cities, of which: 41 with more than 500 thousand inhabitants, 278 with 100 to 500 thousand inhabitants and 337 with 50 to 100 thousand inhabitants. To see using this link.

[iv] Prepared in collaboration between IMAZON, the AVINA Foundation, other entities in the Amazon, and the Social Progress Imperative (first prepared in 2024 and expected to be updated annually). Its objective is to measure results in social development, not just investments, to assess whether the population's basic needs are being met. It is composed of 53 secondary indicators from public social and environmental sources that measure results, not investments. These variables were aggregated into a general index, with a score from 0 to 100, and indexes for 3 dimensions (Basic Human Needs, Well-being and Opportunities) subdivided into 12 components. To see using this link.

[v] The final result is a hierarchy of cities based on six sets of indicators leading to a general result and results by economic segments: Commerce, Services, Industry, Real Estate Market, Civil Construction, Education, Health and Agriculture. To see using this link.

[vi] Produced by the National School of Public Administration (ENAP), which takes into account the most populous municipalities and analyzes seven criteria: Regulatory Environment, Infrastructure, Market, Access to Capital, Innovation, Human Capital and Entrepreneurial Culture. (Data from 2023) See using this link.


the earth is round there is thanks to our readers and supporters.
Help us keep this idea going.
CONTRIBUTE

See all articles by

10 MOST READ IN THE LAST 7 DAYS

SEARCH

Search

TOPICS

NEW PUBLICATIONS

Sign up for our newsletter!
Receive a summary of the articles

straight to your email!