How we make it

What is the purpose of the Annual Report on Annual Reports?

The Annual Report on Annual Reports was created in 1996 at the Enterprise Group, a small Brussels-based international management consultancy that had set up a reporting unit (spun off into e.com later in 1999) to advise companies on their annual reporting process and content (then mainly in the financial sector). Now in its 20th year, the core business has not changed: surveying and benchmarking best reporting practice in order to strive for higher standards in financial reporting, investor relations, corporate communication, and stakeholder information. With higher report value for shareholders, richer report content, better access to and transparency of company information for all stakeholders, increased investor confidence, and decreased cost of capital as subsequent results.

What is ReportWatch?

ReportWatch is the denominator, trademark and website for the report monitoring, scanning, scoring and rating process that results in the Annual Report on Annual Reports, which was first printed and mailed and is now posted online yearly in August. Based both on e.com’s internal desk research and an external panel of reporting specialists, this survey of annual reports’ best practice is widely regarded as the only independent and the most comprehensive, international and authoritative survey on annual reports. (See http://www.reportwatch.net/uploads/files/what-readers-and-users-think.pdf for readers’ comments and http://www.reportwatch.net/best-annual-reports/media/ for corporate and other financial and business media coverage.)

How are companies selected?

The ReportWatch monitoring process starts with the selection of a sample of listed companies –from 250 to 500 when the survey was launched to about 1,500-2,000 for last years’ surveys. Our sources for selection include published international and local rankings as well as internal desk research based on company positions in their industry, peer groups, and past annual reporting records. Many of the large(st) companies are therefore part of the primary selection yet a significant number disappears later due to insufficient report quality. These last years, we have also recorded an increasing number of spontaneous applications from company report makers keen to benchmark their report against best practice.

Though imperfectly -owing to various factors such as lower reporting standards, emerging reporting practices, less developed IR policies, or the lack of report applications- our list of companies and their reports is a relatively representative cross section which reflects the industrial, geographical and stock market diversity upstream, and best reporting practice downstream. The fact that a majority of reports rated and ranked still come from mature economies is thus not entirely our responsibility. Still, albeit slowly, in the wake of the power shift in the global economy, the number of reports from other areas has increased significantly over the past few years.

Our main goal is to be selective and representative rather than comprehensive. Although striving for a sample as representative and large as possible, we easily admit to cover a small portion of the worldwide quantity of listed companies, now estimated at more than 40,000. A survey of all of them would probably be a mission impossible to accomplish.

A ranking is competitive in nature. This competitive dimension appears e.g. in the direct link to a comparable company for each report rated and in a sample of peer groups benchmarked. Every report scanned is immediately compared with a peer. As a result, some industries, companies and reports are left out, while some reports are left in just because they are compared to higher ranked peers. Of course that penalizes hundreds of (un)known companies whose annuals might certainly deserve a good and often better rating.

Note. The name of the company that appears in the Annual Report on Annual Reports is the one as referred to on the covers or as written or abbreviated in key report sections (profile, message). For legibility reasons, legal forms or words such as corporation, company, group, holding, etc. are not reproduced. Except for clarity and communication, names do not take into account mergers, acquisitions or brand identity changes that might have occurred and been approved after the fiscal year-end or the report release.

A focus on listed companies

Was the annual report invented first for listed corporations to report to their shareholders? The answer is not certain, yet is most probably positive. There are hundreds of thousands or even millions of institutions in the world releasing yearly reports, and some of these can be very exciting –a number even much more compelling than compliant or puff pieces from the private sector. Even though U.S. Steel is often referred to as the first certified annual report, published in 1903, "in 1959, IBM hired Paul Rand, a prominent book designer, to create its annual report. As a result, the high-concept annual report was born." (according to the Addison Annual Report Handbook 2005). As for any survey, a scope has to be defined. Since its inception the Annual Report on Annual Reports has therefore focused on reports from listed firms. (°) These can be compared more easily, and this is even more the case due to the effects of globalization, permanent access to information channels (and internet), and the extensive application of international accounting and reporting standards.

Though the selection and the evaluation criteria remain primarily based on stock-listed companies we leave the doors open to any company who wants to submit its annuals for rating. That explains why the readers find an increasing number of less known companies -including privately or government-owned firms (small or larger)- in our ranking –some of them producing annuals that rival with, and sometimes outstrip listed firms’ documents.

Reports for a fiscal year ending any time in the year 2015 (February 2016 at the latest) were considered.
Were not considered for selection:

  • Reports for a fiscal year before or after 2015 or interim/quarterly reports.
  • Financial sector companies (banks, insurance, investment funds, financial holdings) (see below);
  • Privately owned companies (except those electing to compete);
  • Purely government-owned companies (except those electing to compete or those compared with);
  • Wholly-owned subsidiaries (except those electing to compete);
  • Investment, income, mutual or real estate funds and trusts;
  • Listed stock exchanges;
  • Central banks;
  • Development or reconstruction banks and similar financial institutions;
  • Public agencies;
  • Non-profit organizations from any sector.

A line has to be drawn somewhere. The above are not included in our survey due to the inapplicability of a significant part of report evaluation criteria based on listed companies, as well as to various comparability and consistency reasons (a story of apples and oranges).

(°) Note. As a consultancy, e.com regularly advises privately owned and publicly (or government) controlled institutions.

Why are reports from the financial sector no longer rated?

For consistency, comparability and credibility reasons, it was decided in 2009 not to select financial sector institutions for the Annual Report on Annual Reports, i.e. annuals for annus horribilis 2008. Note the premonition: for reports of the previous year (2007), best practice in risk reporting in the financial sector was already “intentionally left blank”. The large number of repeated incidents, crises and malpractices in the banking and insurance industries these last years question the input and output of reporting and would make its subsequent evaluation difficult -and risky! We have therefore stopped watching reports from the financial sector ever since.

That does not imply that there are not (very) good -and improved- reports in the sector, such as some who ranked -some of them high- in past competitions. Outside ReportWatch, with its solid track record in the assessment and benchmarking of reports in the financial sector, e.com keeps on providing evaluation services to some financial institutions that keep on striving for higher reporting standards and best practice –and there are still some.

What is judged –the company or the report?

The scoring, rating and ranking are based on an evaluation of the company report and output and cannot be interpreted as such as an assessment -and even less a rating!- of the company that releases the report. Put plainly, ReportWatch scans the how and, to some extent, the what is reported and not as such the who and the why. As our cautionary statement puts it, the Annual Report on Annual Reports does not represent directly (what about indirectly?) an offer to buy, sell, hold or trade the securities to which the reports cited or ranked in this survey are related.

That said, investors, especially long-term ones and other stakeholders, might infer some opinions and decisions based on report content for last year and consistency in reporting over a period of time. (°) After all, shouldn’t a company who treats its current shareholders, potential investors and other stakeholder audiences well, not least through good reporting practice, deserve more market confidence than others? 

Even though a relationship may sometimes be found between company, report and shareholder value, talking of a correlation would be excessive. Consider e.g. the following points:

  • Before the 2008-… financial crisis, some of the financial sector institutions that later showcased malpractice stood out among report good or best practice;
  • Good performance does not necessarily translate into good reports. In the middle of the worst financial crisis in decades (see: Annual Report on Annual Reports 2009), almost 60 percent of annual reports rated A were made by companies having posted decreased profits or significant losses;  
  • Company size or reputation does not mean good reports per se. All over the years some among world-famous companies have never been capable of publishing high-quality annuals (take Apple and Microsoft among many examples). While others, especially in North America, have moved from good quality annuals to pure compliance exercises resulting in illegible and dull documents (the list of bad examples would be too long to publish);
  • Lastly, a good report doesn’t necessarily equal a “good” company –if we may put it so (with all due caution, that is): 8 companies from our 1997 top 20 (i.e. 40%) have vanished, stopped, or been taken over since then (the list includes Enron), but believe us, some of their reports were really worth a read! 

(°) For tracking historical annual report performance, use our ranking index available at http://www.reportwatch.net/best-annual-reports/a-z-ranking-index/.

Is report entry free?

Bar the above-mentioned restrictions, any company may submit is annuals at any time.
Participation in the survey is entirely free of charge, except of course for mailing, downloading, copying or printing costs incurred.
Naturally, a report submission does not automatically guarantee rating and ranking.
The use of e.com report evaluation services is no prerequisite to -and no guarantee for- being selected, rated and ranked and is independent of the ReportWatch process and the results as published in the Annual Report on Annual Reports. We view that as a guarantee of neutrality.

Which documents are assessed?

Documents named, linked to and referred to as “Annual Reports” are assessed, as well as summary versions (reviews, overviews…). Corporate social responsibility (CSR) and sustainability reports are evaluated, either in printed, PDF or online format, as a component of annual reporting. Proxy forms or separate governance documents are considered based on their availability.

Reports simply made up of a legal file (e.g. 10-K, 20-F or other similar GAAP and proxy forms) are only considered as eligible -even if they fail to match a significant number of our evaluation criteria- when they are compared to more elaborate reports. That explains why a number of basic reports are ranked and, on the other hand, why a number of well-known blue chips who stick to purely legal reporting forms do not qualify for being rated and do not appear in the ranking.

Printed or online reports?

The increased use of the Web as a corporate communication and investor relations channel has to be reckoned with. Compared to two decades ago, when a small 10% of rated annuals were going for partly web-based reporting, most of the ReportWatch process is now based on HTML reports or PDF versions downloaded from corporate websites. However, connecting the dots from an online report can sometimes be as difficult an exercise as it was ten years ago. Printed copies, PDFs, printouts or effective e-books are still preferred by many -and by our team- when it comes to in-depth report screening. When an online report is judged as optimized for reading, the scoring and rating is based on it. In other situations, and these still constitute a very large majority, PDF, e-book or printed versions are scanned and scored.  

The ReportWatch criteria are based on report content and apply whether published on paper or on screen. The investor, analyst, stakeholder and any reader should find the information required by regulatory bodies as well as what the company makes available beyond compliance whatever the mode of communication. Except for communication and a few specific aspects, all evaluation criteria (see below) apply in both printed (or PDF) and web (or HTML) contexts.

Does that imply that a digital report should simply be a copy-and-paste of a printed one, or vice versa in the near future? Certainly not. Corporate and investor websites can be used to (re)format, (re)structure, (re)design annual and other reports, and, in best practice, to add value by providing extra features and contents for stakeholders. These aspects are naturally taken into account in our scoring and rating job.

Which evaluation criteria for what results?

Report scanning and scoring is the first stage of the ReportWatch assessment process. It is carried out by e.com report analysts (financial analysts, investor relations specialists, corporate communication advisers, accountants, economists, copywriters, at senior and junior levels) and provides a basis for final ratings by the rating panel. Ultimately, it results in the report ranking published in the Annual Report on Annual Reports.

The globalization of markets, round-the-clock cross-border investment flows, internationalization of reporting standards and their implementation, and the increased complexity of reporting requirements have resulted in an overall improvement of reporting practices around the world and a larger degree of homogenization of annual reports over the last decade.

Those trends have made the ReportWatch evaluation and scoring job more difficult. In the first five years of the Annual Report on Annual Reports two-thirds of top 100 reports were rated B or lower on key reporting areas. These last years, about 60 percent of top 300 reports have been rated B+ or higher. Despite a gap between lowest marked reports and top ones almost as big as a decade ago, there is now a 25-mark difference between top 10 annuals and the ones ranked around 250-300. That is much, or a little, depending on the viewpoint.

In the first five years of the Annual Report on Annual Reports two-thirds of top 100 reports were rated as a whole B or lower. 52% of 1996 top 100 annuals were rated B+ or higher. Two decades later, not only the whole top 100 scores B+ to A+, but more than 70 percent of top 300 reports were rated B+ or higher. If the gap between lowest marked reports and top-rated ones is almost as big as ten or twenty year ago -there is still a 30-point difference between top 10 annuals and the ones ranked at the bottom of top 400 (for reasons that may differ)- it is worth noting that the difference among top 100 reports is about 15 points, which is much lower than twenty years ago.  The trends mentioned above and their impact on report input and output make the ReportWatch evaluation and scoring job more difficult.

Report evaluation criteria have to cope with trends and challenges. As a result, they have evolved and are updated and upgraded regularly. For those who would have forgotten, governance, compensation, social responsibility, sustainability, integration… were hardly on the reporting agenda two decades ago. Remember the rise and fall of Enron (another “Wall Street darling” -and an annual report considered by our analysts and panelists good enough to reach No. 11 in 1997). The net result of this and other scandals was Sarbanes-Oxley and other related regulations, whose impact on reporting and communication was far from being always positive.

Modifications in evaluation criteria may sometimes impact on report score, rating and ranking. Although e.com-ReportWatch emphasis has always been placed on financial, business, strategy and performance reporting and investor indicators, the report assessment criteria have consistently been based on a well-balanced perspective blending financial and business analysis; short- and long-term performance aspects; strategy and operations; visual and textual elements; share- and broader stake-holders’ issues; information content and communication style –whether in print or online.

For the annual reports for a fiscal year ended in 2015 (i.e. between March 2015 and February 2016) five reporting areas were evaluated: communication & style; operations & sustainability; strategy & leadership; figures & financials; investors & governance. The following report items were scanned and scored:
Online navigation – Cover and introduction – Profile/Overview – Key figures – Message, identity, branding – Key performance indicators – Business model, value and strategy – Outlook – Goals and targets – Business report – Financial review and management discussion – Risks – Investor information – Share items – Sustainability and integration – Social and environmental impact – Leadership and management – Governance – Compensation – Statements and notes – Charts, diagrams, maps – Style, layout and design. 

Are the report marks made public?

Only the ratings are made public. In line with our tradition since the launch of this survey, the total score or its breakdown is never publicly disclosed. It may be obtained by companies or via their advisers/agencies through an order for a Report Scan (an edited output of e.com’s internal desk research for the Annual Report on Annual Reports). (°) The revenues generated through scans and other evaluation services help us produce the Annual Report on Annual Reports –and keep it independent.

(°) Scans are among the numerous evaluation services that enable e.com to publish a self-financed survey based on totally independent research. Go to http://www.reportwatch.net/e-com/making-reports-pay-off/  for more information.                                 

How are reports rated? What is role of the rating panel?

Based on the marks resulting from e.com’s scanning job, an internal rating is given to reports.
Ratings and rating agencies have drawn a lot of criticism these last years, mainly due to the use and misuse of measurements, questionable accuracy, misjudgments, etc. It is worth reminding that, even when based on objective assessment criteria -what we are trying to achieve with our methodology- ratings are also often made up of more subjective judgments and perceptions, and cannot completely exclude bias. Our report ratings should therefore be seen as indicative -neither more nor less- and not be considered as such as an opinion about the companies/stocks/investments’ past and future performance.

The primary role of the independent rating panel is then to cross-check top reports scored by e.com and to help move from a very quantitative and “dry” scoring to a more qualitative rating, based both on intrinsic report value and communication towards various investor and stakeholder audiences. As a result, some reports are upgraded while others are marked down, from slightly to significantly. Up to one third of top 200 reports may move up or down in the ranking after the rating panel’s intervention. Because of rising reporting standards -the number of reports rated B+ and above has doubled over the last two decades-, better practices and increased homogeneity in requirements, the rating task is much more challenging than in the past.

Panel members are appointed for their experience and expert knowledge in corporate reporting, financial communication, investor and public relations, and any matter related to report publication and content (see separate section for panelists’ profiles). The ReportWatch rating panel, made up of all-round and specialist members, has always been characterized by its diversity (see http://www.reportwatch.net/best-annual-reports/report-rating/).  
Panel members have to judge independently of their own interests (they may not rate reports in which they have been involved or have a vested interest). Individual votes are not publicly disclosed. The final ratings and ranking as published in the Annual Report on Annual Reports are the sole responsibility of e.com - ReportWatch. 

How is your report doing?

How is your report doing? The Report Scan?

What is your report rating? How does it score –in total and on all evaluation criteria used for the Annual Report on Annual Reports?
Order an ANNUAL REPORT SCAN. An edited output (°) of desk research done by e.com report analysts, it provides your company (or advisers) with the score breakdown for 25 report items and a summary of pluses and minuses for each of them.

The price? £ 900 or € 1,100 or US$ 1,500.

(°) For copywriting and editing reasons allow a few weeks for Scan delivery.

E-mail your order to: e.com@reportwatch.net