One issue that is increasingly rearing its ugly head is the ridiculous – and often insane – demands industry analysts are placing on providers of technology and business services to pony up client references for their scatterplot charts. The situation has become so bad that the integrity of these research processes appear to be reaching a breaking point, and I would argue that some form of regulation is needed to protect the interests of the business consumer.
The leading analyst firms are demanding five client references per provider.. and one recently even requested TEN client references. The requests are made with the veiled threat that the analyst will “not have sufficient information on the provider’s performance” if these references aren’t made. When I repeatedly have multiple providers complain about the situation to me, in addition to several of these overused buyers, surely it’s high to get this issue on the table?
Correct me if I am wrong here, but isn’t it these analysts jobs to have regular ongoing conversations with buyers of their covered markets, so added references are merely a rubber-stamp? In fact, why are references even needed if these analysts are so informed, connected to the buyers and have so much valuable data and research to call on?
So why is this a growing problem, I hear you cry?
Can you imagine what it must be like for a provider to ask a good chunk of its referenceable clients to partake in one hour reference calls with 3, 4 or even 5 analyst firms? Not only that, these same clients are being asked to provide even more detailed references to sourcing advisors and management consultants for their procurement and vetting processes. Hence, some of these referenceable clients are being asked to spend many hours of their time talking to analysts and consultants about the same old stuff. I have had several providers openly complain about the strain this is putting on them – and some are almost at the point of simply having to decline to provide any, as their clients are literally fed up with the whole racket and the demands on their time.
While these analysts can argue that providers should have lists of hundreds of clients they can use for referenceable interviews, when we get into emerging areas like analytics, mobility, social, digital, industry-specific processes etc, some providers will only actually have a handful of clients. So going back to the same old well time and time again to reach those outer peaks of excellence in analyst graphs is becoming an increasingly impossible task that has now reached breaking-point.
Why this dog and pony show needs to change
1. The providers better at schmoozing their clients to pony up references win. Some providers invest a fortune in their “red carpet” clients to ensure they will spend a good few hours each week singing their praises to the analysts. This is nothing new – and most smart vendors have figured out how to game the system over the years. However, where the demands on the clients time increases form 3 hours and month to 3 hours a week, the level of schmoozing required reaches new levels of tolerance, and even the cleverest of providers are struggling to stack the deck in their favor – they can’t provide enough World Cup tickets or VIP parties at the Masters to keep everyone happy. But this also brings into question the impartiality of these references – everyone knows they are going to be given through pink-champagne-tinted lenses, so why some analysts even bother with some of them is beyond me…
2. Some analysts clearly do not spend much (or any) of their time talking with buyers if they are relying purely on provider references. To the defense of the analyst firms, why should their analysts have connections with enough buyers in their covered industry to conduct credible assessments of provider performances? They’ve clearly been making a ton of money employing analysts who live in a world of listening to vendor hype, so why should they correct-course and make sure they have their analysts spend more time in the real world of (gasp) talking with buyers? If the provider side keeps funding these analyst firms to keep behaving in the same way, nothing will change and this situation will continue to worsen.
3. Some analysts don’t even bother to include providers in their scatterplots if the vendor refused to “partake”. Yep – this happens, folks. How any credible analyst worth her/his salt can put out provider assessment scatterplots and leave out major players in their market is beyond me. However, there are analysts who literally have no other way to “assess” provider performance if they do not have heaps of marketing guff and a few client attestations to lean on, so they just drop those vendors unwilling to pony up the references and “partake” in their assessment process from their report.
The Bottom-line: The analyst business is caught in a vicious Catch-22 cycle and the only way to break this is to regulate it
Have some sympathy for the providers here – there are clients who make purchasing decisions based on analysts’ assessments, so they have no choice but to jump through the ridiculous hoops some analysts are demanding. Not only that, these vendors are forced to help fund these analyst firms so they can ensure they can monitor the analyst research and get enough face-time with the analysts to get them to portray them as accurately as they can hope. So while the beast is constantly fed, this situation will only continue to exacerbate.
In a world where equity analysts can’t even have a provider buy them a diet coke, why are so many of the industry analysts allowed to perpetuate this ridiculous racket of having providers fund them, and even do their “research” for them? Surely when an industry reaching a quasi-monopolistic situation, like the industry analyst business has, it’s time for fair play to be demanded?
The industry for technology and business services alone is over a trillion dollars annually (and that’s just external spend, not internal). Surely it’s time for government to step in and demand these “research” processes are run fairly, and these analysts are doing their research appropriately? Doesn’t the government have a duty to protect its business consumers to ensure they are getting far and balanced advice for critical business decisions that could significantly impact their futures? Should firms peddling “research” need to to meet standards of quality and integrity to which so many other industries are held? Somehow industry analysts have escaped the real world and create their own rules and economy…
Posted in : Business Process Outsourcing (BPO), IT Outsourcing / IT Services, Sourcing Best Practises
Refreshing insight, Phil. Always enjoy reading your views,
Heather Keene
Phil,
I agree there needs to be some form of controls over analyst processes. I am not sure how they would work in practice but there clearly is a need for consistent quality and less bias from many analyst firms,
Abishek Jain
Bravo Phil – love your rants!
Completely in agreement here – especially with the client referencing. Analysts need to form their own independent relationships to get the “real” view points form clients,
Al
Phil,
Excellent insight, but I do not know how regulating analysts would work, and not sure we need to spend more tax dollars on policing even more industries. I do believe that some form of accreditation is appropriate, such as an independent review of quadrants that assesses the quality of client references, the capability of the analyst and the overall methodology deployed.I believe this is especially needed for services areas where provider performance is very execution focused,
Colin
Phil hits another one out of the ballpark! Super insight as always.
Phil,
The intent is good. Legacy analyst firms are often too lazy to find their own buy side clients and the pay to play independent analysts are also short on end users. In a world where no one wants to invest in buy side clients, it’s getting harder for vendors to see the value of analysts. However, I think more transparency on the bad practices of legacy analyst firms and a concerted effort vendors to snub their nose at the legacy analyst firms may be a better model. Of course they could also support the analyst firms who are focused on building a buy side base.
Market forces will correct themselves, even if it seems abysmal right now. We just need more transparency on the issue. Also other ethics should be looked at too, but you’ve already known this.
R “Ray” Wang
Principal Analyst, Constellation Research, Inc.
Hi Phil,
As someone who has worked on the vendor side for many years, I couldn’t agree more. Some analyst’s work ethics are very questionable and the results of this ‘research’ are sometimes dubious, more often laughable.
I even have had some analysts that inform me by email that they are planning to do a report on trends in our industry, it usually goes something like this – We are delighted to let you know that we have recognized that you are a leading vendor in the xxx market. Please kindly fill out the attached 20-page Q&A and put all you’re shipping numbers in the attached spreadsheet. Please complete the task by the Friday deadline or unfortunately you won’t make the report.
So, I agree 110% with the gist of this article, something definitely needs to change. Maybe industry self-regulation or certification is the way to go, like CPAs or the Law profession. Either way I would love to stop wasting half my time questioning the validity of the research/advice that I am getting and just get on with my real job of making decisions for my company.
Jonathon
@Abishek – the analyst business needs to self-regulate for the long term health of the business IMO, or the credibility will continue to erode over time.
@Colin – without the blatant sales pitch, the HfS Blueprint is designed to rate providers on execution versus innovation, and weight the performance against criteria that is important to the industry stakeholders (buyers, advisors and analysts) – you can download the methodology here. The analyst weight is only 10% of the final score. Too many of the legacy analyst firms shoe-horn the services provider ratings into the same criteria they use for software products – which are largely irrelevant.
PF
@Ray: “more transparency on the bad practices of legacy analyst firms and a concerted effort from vendors to snub their nose at the legacy analyst firms” – you basically said what needs to be done. While people like you and me can highlight to industry these poor practices that often lead to misleading information and unbalanced analyses, only the vendors who fund these firms can do the real talking – by walking away from them. While we’re seeing some evidence of change occurring in analyst circles (i.e. Forrester’s new attempts to become more of a strategy consulting firm), the same old practices of lazy analysts relying on vendor references to do their “research” are seemingly only increasing in intensity. Based on the feedback I have been getting recently from several services vendors, many of them are at breaking point trying to jump through the analyst hoops and may be close to walking away. Ultimately, it all depends on the credibility the buyers still place on the scatterplots. As the CEO of one major vendor told me recently, “I don’t care how far to the right hand corner we are, as long as we are in the right hand corner”…
PF
Phil,
Good insights – and provocative too! You are correct about this Catch 22 cycle. To sum it up, vendors pay analysts to access their research and to try to influence it. Analysts are lazy and demand vendors do their research for them by completing their forms and setting up their client interviews. Vendors get upset because they are overloading their clients with analyst demands and users are disappointed with the quality of the research. The answer is simple – the vendors need to stop funding the analysts to perpetuate this cycle. That is the only way to break the cycle and improve the quality and integrity of the research. Money talks.
Eric
As mentioned a long time back, here’s the scenario and why it’s hard. Vendors and analyst must agree not to engage in the bad practices.
http://blog.softwareinsider.org/2013/08/09/personal-log-the-sad-state-of-the-industry-analyst-business-and-the-need-for-a-code-of-ethics/
R
I didn’t need references or vendor data when I wrote my analyses. I could have written any analysis without…but the vendors better fear that. I’m sure the same precious few customers get the nod to help because the rest hold average opinions of their vendors. If I did this, I bet the whining cacophony would have been deafening (Who did you speak with? That had to be one bad apple! We have better clients-did you talk with…?). I’d suggest the vendors quit supporting the analysts that don’t have the reach and buy-side relationship.s that create influence Who cares if an analyst with no standing working for a sell-side-focused analyst firm wants you to spend hours gathering data? It’s a waste of time. Pick what matters and make it great…
That said, maybe it’s time that analyst firms either stop taking money from vendors or they disclose which vendor is a [relative size] client. The financial ones do that…firewalls and disclosures
[…] This post from HFS that I read over the weekend poses a great question about regulating the Analyst Industry. It reminded me of an incident I had with an analyst a while back…. I had worked rather diligently with particular analyst from a ‘reputable firm’ to assist her in delivering a balanced view of the market (over a number of quarters). I had worked hard at the relationship, making sure we were providing as much value as we were (presumably getting). In the lead up to her semi-annual market update, she scheduled an interview and sent over an excel. I dutifully filled out the tables and sent it back and we followed up with in-depth, interactive conversation over the phone. Considering we had two great quarters for the coverage period and the market had shown some good momentum, we were very upbeat an eagerly awaiting the report. […]
Hi Phil,
thanks for this, I understand the problem, but we are far from getting an interventionist solution. Government regulation is going a step too far. Does the US government have the right to regulate the global analyst industry. How do you regulate an European analyst firm getting a reference from an Indian vendor for a market in Africa. If US government intervention looks like health care, gun control, or financial services, is it something we really want for our industry.
The answer in my humble opinion is therefore not government regulation but some form of inertia in the industry evolving to a more ethical and reasonable environment. We will get there, but there is too much fear by vendors in upsetting Gartner.
As a fellow disruptive (in my own way) independent analyst we are in for the long haul, change when a market is dominated by a provider who is about 4 times larger than the nearest vendor is difficult, but can accelerate over time if we have a bit of luck individually and collectively.
@phil – my recommendation is self-regulation from the analyst industry itself as a first step. I do not agree that the industy is “getting there” as many of these shoddy analyst practices, some of which I have highlighted, are only deteriorating further.
PF
Phil I did not say we are getting there, but that we can. I try to always be the optimist.
I think most industries have shown the problem of industry self regulation, it does not work, not does government intervention in these issues, particularly with the geographic diversity.
Geographically i am removed from some, though far from all the issues. If the system is in such decay then the renewal will happen at an accelerating rate. Holding out through a time of pain to help reinvent an industry sounds like a valuable proposition.
Phil, This PoV is definitely a need of the hour and should be circulated among entire analyst community for all to introspect the way we have got used to obtaining and then projecting information. Are we becoming someone that the vendors would rather avoid?
On the contrary, anything that is governed in the free-speech system loses sheen, so who will look at governed 2×2???
Oh this article has made my thoughts run in all the directions….
Long overdue public discussion! Agree with so much here about the ideal of vendors not funding analysts (but, c’mon, vendors will buy advertising in whatever format they think will help). We also REALLY need industry action toward holding analysts accountable for publishing where their revenue comes from – like to at least know percentage from vendors vs. subscribers/users. Final comment that no one has honed in on yet — we need to keep this type of dialogue flowing and bring it mainstream. There is a great need to help buyers “beware” and educate the corporate buyers on how the analyst industry works and what methodologies and ethics to look for before they succumb to high priced annual membership dues or outrageous prices for annual canned reports (thereby also feeding the industry).
Phil,
Great article. I did a similar rant based on my experiences as an ‘industry analyst’ at our alma mater AMR (Gartner), although I went off the deep end a bit… http://spendmatters.com/2014/05/28/toothless-bumbles-and-the-search-for-objectivity-in-the-procurement-solutions-market/
Like the other commenters, I think that markets are generally efficient and don’t need external regulation, but we do need better self-policing, especially as it becomes more difficult for practitioners to decipher objectivity as vendors begin to use ‘analysts’ that are anything but. I described these various sub-segments in my rant above, and it’s a spectrum that moves to pseuod-analysts, white paper firms, bloggers (which is dicey term – blogging is something you do – not a definition of the services/value you offer), freelancers, and payrolled journalists. It’s interesting how some of the big vendors will surround themselves with pseudo-analysts when even the big IT analyst firms can’t be bought off (er, I mean, convinced). It’s like those bad movies that have trailers with the glowing quotes in 96 point font and then the 6 point font of the reviewer being Johnny Bumblefukk from Wichita affiliate of CBS (no offense to Wichita). Client disclosure would be a good start. Maybe include such symbology into the quadrants/waves.
If we don’t police it ourselves, then it becomes a wild west. Sort of like LinkedIn groups where half the ‘discussions’ are re-posts by ‘social digital media strategists” to the websites of their employeres. sigh. But that is a rant for another day.
Keep up the great work – and maybe we can convene a ‘Hall of Justice’ to work on such self-policing policies. Just don’t make me Robin to Jason being Batman. It’s not the tights or anything, but, well, never mind. First order of business will be drinks and then nominating the members of the Legion of Doom.
[…] Research Services « Is it high time industry analysts are regulated? […]
Great article. As someone who began as an HR practitioner then moved to working more heavily in the HR technology field, it has been an eye opening education for me to learn about the unseemly underside of technology consulting and industry analysis. The buyers “trust” the industry analysts and do not realize how much of data is manipulated and influenced. The reputation of the industry analysts has been built on a foundation of sand but that foundation is still presenting sufficiently well with the buyers. Buyers often succumb to the fear that without using name brand resources, they endanger their credibility. Firms like my own that struggle to remain independent and objective, also struggle with cashflow as our voice is drowned out by the marketing noise of the brand names. In my experience, the best source is the community of buyers – but they are also the very people who succumb to the song of the sirens.
[…] if by some freak of nature, the very next day after we stirred the pot questioning whether analysts needed regulating (or self-regulating), the industry’s leading analyst/influencer observer, the venerable Duncan […]
[…] the current FIFA scandal, it’s interesting to note that it is a year since the most serious call yet for the regulation of industry analysts. Phil Fersht argues that the growing demands placed on […]