There’s nothing more jarring than an ex-Gartner analyst desperate to continue dining off a legacy analyst industry that is actually trying to change. And lo and behold, Just before the Christmas break, a blog emerged on LinkedIn with an enviably click-baity title ‘Is Gartner research quality under threat?’.
Simon Levin, a Gartner Alum and owner of a boutique business “The Skills Connection” that helps tech vendors lobby their way through the Gartner and Forrester MQ and Wave processes, plies his trade on the fact he “knows” how to work his friends at Gartner, to help his vendor clients get their dots edged in a more positive direction for the firm. And why not? If I was a CMO, and lobbing Simon some moolah can help get some sort of leg-up in the process, I’d probably give him a shot. And however you performed, there is no doubt Simon will claim it would have been worse without him. It’s like that hair product “Rogain” that claims to slow down hair-loss… you’d never really know if it actually helped unless you went completely bald…
Simon makes the case that now most his former Gartner buddies he worked with in the 90’s are being quietly “retired” and replaced with a new breed of youthful analysts, and their research maybe less predictable:
“Quality control is hard to enforce. When a client of ours became involved in escalating a dispute over an MQ assessment recently, we saw some signs that Gartner’s controls may occasionally creak at the seams.
Our vendor noted factual errors in both the wording and the scoring of the draft assessment. We protested and took the client through the escalation process, and the company’s dot was rightly promoted from midway down Niche to the top quartile of Visionary, while the words were rewritten and became significantly more positive.”
So Simon literally cannot lose here: “In the old days I could leverage my influence with my former colleagues, now they’re going, I can help even more with the new kids they’re throwing into the mix who might make a few errors”
Predictably, the conclusion from Simon Levin was “as long as you hire my firm early enough in the MQ process to make sure their analysts don’t misrepresent you, you’ll have nothing to worry about”.
Why the content of Simon Levin’s blog should also be of real concern to the industry
In many ways, the stance taken in this blog is representative of how tech marketeers AR professionals view analysts – after all Simon’s revenues come from advising vendors on how to put pressure on analysts in the right places. But let’s break down in stages what’s wrong with this argument:
Misguided principle 1: Wining and dining analysts mean they’re in your pocket and should stay there!
The piece starts off innocently enough by confirming what we already knew and discussed in our blog covering nodding dogs – vendor marketeers and their analyst relations professionals should expect analysts to do their bidding all the time they’re being wined and dined. We know this because Levin discusses at length how irritating it is for vendors when they have invested so much time “cultivating” an analyst, to find out that said analyst isn’t handling that big competitive analysis this year. The misguided presumption here, of course, is that wining and dining analysts is just as important for getting a decent score as, say, providing relevant and timely information.
While this isn’t true for many analysts – we can infer, given the provenance of Levin and his compatriots, that Gartner analysts are perhaps more willing to boost scores if they have a good relationship with the vendor. Isn’t this knocking the value of objective assessment somewhat? I’m sorry Simon, but most analysts do this job because they actually value their ability to be objective. While one analyst firm obsesses with incentivizing its analysts with P&L responsibilities, most separate their analysts from the direct revenue impact of license reprints. Having a good relationship never hurt anyone with any business engagement, but a decent ethical analyst is never dissuaded by a decent steak (or vegan) dinner. Which moves us neatly on to our next puddle of misguidedness…
Misguided principle 2: Fresh perspectives are bad – the game is about controlling stale opinions, not embracing new insight
This is a big one, and it slithers through Simon’s blog as a core theme – the only stakeholder worth thinking about is the vendor. What’s misguided here, though, is the prospect that a fresh analyst jumping on a piece of research is always a bad thing. Sure, vendor marketeers and their analyst relations professionals may be rightly upset the steak dinner they bought has gone to waste as a new analyst’s in town, but for the enterprise executive who relies on balanced research to make decisions, a fresh perspective is almost always welcome.
The same staid analysis from a crusty ol’ analyst recycling the same themes may well be predictable and easy to influence. We all know the types who have those decade-long loyalties to the likes of SAP, Workday. Oracle, IBM et al. It’s very hard to convince a 30 year long analyst that the world is changing and and the vendor that have been lauding for their entire career may be losing its edge. Changing things up a bit is important to keep the research fresh, and analysts on their toes.
However, change is nearly always inconvenient for vendor marketeers and their analyst relations professionals – the clientele that keep Simon Levin’s coffers swelling. What’s soul-destroying here is that the quality of the research (despite being the core focus of the misleading title) is pushed to the fringes of the discussion. Simon’s blog isn’t about getting the best research and coverage into the market. It’s commiseration on the annoyance that is losing a tame analyst for a new one that may not be as willing to down the Kool Aid.
Nevertheless, quality is mentioned – but it masquerades in the narrative as a single metric – the average tenure of an analyst. Because, of course, the longer an analyst is in a firm, and covering an area, the better they get. I mean that’s just math’s…sort of. But unfortunately, even that old-world idea is being consigned to the history books. And – just like the rise of avocado ownership, and the crippling private rental market, we may as well blame the millennials for our next part.
Misguided principle 3: Experience trumps relevance
This is the bit that really goads us. Why are we perpetually locked in this bizarre world in which we believe the best perspectives come from those that have been doing it the longest? Why do we shrug off the new and refreshing insights because their origin hasn’t been dipped in decades of painful CIO workshops? We endlessly hear the need to develop new skills, to re-imagine our processes and redesign our business models, so why do the analysts get a free pass when it comes to keep up with the times? Why shouldn’t analyst firms practice what they preach?
The main assumption in Levin’s blog is that age begets wisdom. That there may be a quality issue at Gartner because the average tenure of analysts has dipped. But could it be possible, just possible, that the average tenure dropping is a good thing? Already we’re seeing hordes of younger executives and professionals flood into enterprises, no longer restrained by mandatory experience quota’s or ridiculous policies from HR insisting that employees stay put in a junior role until they’ve paid their dues. This coupled with the army of young entrepreneurs driving growth in tech start-ups and disruptors, is starkly altering the demographics of the average analyst firm’s client. So, is it reasonable to infer that the demographics of the analyst catering to this market also evolve?
Comments and blogs that imply quality research is inextricably linked to the quality and value of research are not only wildly out of touch with the changing market. But they’re just obscenely ill-informed. In a room full of 20-year-old tech innovators, does the experience of an analyst with decades of early 90s CIO experience more valuable than a young analyst who can live and breathe the unique challenges of the market?
Bottom Line: It’s not about tenure, it’s not about gaming the system, it’s not about wining and dining analysts. If this analyst is to survive to see 2020, it needs to focus on the quality, relevant research that a diverse market needs
The reality is that diversity is always beneficial and throwing in the perspectives of both analysts will undoubtedly be of benefit. But what doesn’t help is commentators like Levin churning out the same vapid and misguided tripe, in a bid to form a reactionary guard against the changing tides of the market. If anything, they’re proving themselves to be irrelevant and damaging to an industry that, much to our lament, is crashing around us.
Posted in : Uncategorized
Phil, Phil, Phil. So many things to discuss here. I haven’t counted words but I’m afraid my response will be longer than your original post. (Not surprising; I was never very concise.)
First, let”s get the snark out of the way:
– You make Simon the focus of your vitriol. It sounds awfully personal rather than focusing on the substance of your discussion. Very unbecoming of you.
– I find it very amusing that you call things self-serving and clickbaity. Pot calls kettle. And of course it is. Isn’t that what 97% of blog posts and LinkedIn postings is all about. And like it or not, it works. https://www.cbinsights.com/research/team-blog/newsletter-headlines/ I’m sure you posted a huge image of Rogain [sic] because it’s closely aligned with the analyst industry.
– It’s amusing to me that you criticize our business while demonstrating that you don’t remotely understand or appreciate our value proposition to our clients (who, interestingly, are *not* for the most part AR professionals but instead are CMOs and, not infrequently, CEOs). Yes, we use our analyst experience and the analyst evaluations as an entry point but our real value proposition to our clients is not “we know how to play the game” (though we do) but instead, “we can better help you tell your story to your key constituencies, including customers, by focusing on evidence and differentiation.” Our very best clients are those who, through the evaluation process, come to realize that telling their story effectively to analysts (the most challenging audience they’ll deal with) enables them to better tell their story to customers, funders, and other essential stakeholders.
– It is perhaps self-serving for you to constantly refer to the dying legacy analyst business model. I hope my death is as, um, robust as this: https://www.statista.com/statistics/722765/worldwide-gartner-annual-revenue/ . And despite your lobbying during analyst influence survey evaluations, Gartner is consistently a strong #1. You’ve proven that you can “punch above your weight” (or is it, as you accuse us of, “gaming the system”) but I’m not convinced the expert model promulgated by Gartner is a dying one. I absolutely believe, and devoted two years of my life, to the belief that there are other models, as exemplified by G2 Crowd, IT Central Station, and TrustRadius, where direct user access will only amplify the impact of peer reviews will produce actionable insights for users. And to the extent your business is a combination of analyst insights informed by deep user insight, bravo for you. And yes, Gartner will have to, and has already, responded to this through their Peer Insights efforts. Is it deeply integrated into their processes? Not yet. Are they market-leading? Not yet, nor would you generally expect an incumbent to be market-leading. But are they making significant changes year over year? Absolutely. So much for the dying legacy model.
Having gotten that out of the way, let’s turn to the substance of your argument: Duh!
Are most analysts reputable and honest? Yes.
“Misguided principle 1: Wining and dining analysts mean they’re in your pocket and should stay there!” Nowhere do we suggest that wining-and-dining is a difference-making approach. Your assertion in that regard is disingenuous. What’s important is for the analyst to have a full understanding of your real differentiation, and I’m shocked by how many vendors do a poor job of this. One of my favorite exercises is to pull quotes from a vendor’s website and from their competitors’, and ask them whose solution they represent. It constantly amazes me how few people can not only not identify their competitors but also can’t even pick out their own b.s…er, messaging. At the same time, to say this isn’t effective is demonstrating self-unawareness. If you think clients are paying for your brilliant insights alone, you are blind to what’s going on. Vendors absolutely believe they’re gaining influence, especially compared to non-clients, when they engage with you. The discussion I had with one Gartner analyst a few years ago is instructive. He posted an emotional post that said vendor money doesn’t buy influence. I observed that either the vendors are paying you too much money (because they think they’re getting influence) or you’re unaware of the influence they’re having. Which is it? Personal relationships do matter, but only to the extent you have a compelling story to tell them. I’ve never had a dinner that was so good as to influence my position. I have, however, had dinners where the deeper understanding of their thought process and capabilities made me understand and value the vendor greater (and also, sometimes less — what a mess!).
“Misguided principle 2: Fresh perspectives are bad – the game is about controlling stale opinions, not embracing new insight.” Can younger analysts be as good as the ones they replace? There are trade-offs and the intelligent organization balances historical perspective and institutional knowledge with fresh blood and different perspectives, especially on new technologies.
“Misguided principle 3: Experience trumps relevance.’ It’s not an either/or. Does experience have value? It absolutely can. It takes time to understand what it means to be a good analyst. There are plenty of industry “influencers” who are terrible analysts. The best analysts have experience which they can apply to current situations while remaining relevant. Can you turn smart people into great analysts? Absolutely, and the best organizations focus on this relentlessly (and Gartner has done this well for a long time).
In summary, I think you make a series of points that are, in turn, interesting, relevant, self-serving, and obvious, while taking a swipe at a person and company you don’t understand. But thanks for the free publicity and the acknowledgment that “If I was a CMO, and lobbing Simon some moolah can help get some sort of leg-up in the process, I’d probably give him a shot.” Can you please put that part in bold?
Yes, I’ve validated your set of “misguided principles.” In fact, I would say I strongly, even violently, agree with them. That said, you shouldn’t view it as a favorable review as you shouldn’t view validation of the ends as endorsing the means. You materially misrepresent our positions as a way of making your points, which is intellectually dishonest at best, malicious at worst.
That aside, vendors are well advised to recognize that analysts serve a very useful role, and remain well trusted in the end-user community (even while many are increasingly reviled in the vendor community). If you can tell your story well and convincingly to the analysts, then you’ve got a great story that you can tell to anyone. And that is the true focus of The Skills Connection, not gaming the system. If only it were that easy…but analysts are just better and smarter than that. Even Phil.
So, let me get this straight – Simon pulls together a blog which takes on Gartner’s hiring and retiring policy with the catchy header implying this was damaging their value. Riddled through the blog were assertions that only those with a lengthy tenure at Gartner could offer value, and clients of the firm should be rightly worried – most of all vendors who would have to drag themselves to a briefing with an analyst they haven’t met yet. We call him out on it. Snarkily (we like snark). You then wordily go through the blog effectively validating the points we’ve made with "duh" or "yeah sure" but then over explain what Simon might have meant – or what profound insight he was trying to impart.
Maybe you should have written the blog on Simon’s behalf? You seem to have a better comprehension of his intended meaning. Like that awful Literature teacher I had that was desperate to find hidden meaning in some, frankly, awful novels. Maybe the author just wanted to say the weather is shite – not that the wind was symbolic of the disposition of the main characters bowels.
In any case, you have basically gone through and validated each of our "misguided principles" so, as the co-author I see this as a favourable book review. So thanks…
Phil (or perhaps I should call you “Fersht” as that is the way you like to describe the target of your rather mean-spirited missive),
I will not attempt to deconstruct the logic of your argument. Jonathan has already done a fine job of that, and you obviously fancy yourself as a master wordsmith who will be savvy enough to attack and discredit anyone else’s opinion.
My concern over your own blog post is more aimed at the sad necessity you feel to attack a fellow analyst. I worked for 20 years at Gartner, and like everyone else I saw no shortage of analyst sarcasm and negativity aimed at my employer on a regular basis. The old adage about paying for MQ dot placements has as much credibility as Donald Trump’s climate change denials. Everyone knows this, and always has, but it’s a nice way to cast aspersions about the 800 pound gorilla in the market. So I get it that everyone likes to hate Gartner – it’s the norm. A bit like hating Manchester United, or Bayern Muenchen, or the New York Yankees. But when I say I hate Manchester United (which I do), I am really hating the INSTITUTION that Manchester United represents. I don’t hate their players, or their fans, or the thousands of decent people for whom Manchester United is an employer or an important economic lifeline.
And during that 20 years at Gartner, I never felt the need to belittle any other analysts. Gartner employs some great analysts. Are their analysts uniquely qualified or skilled – absolutely not. The analyst community is an immensely rich pool of talent and opinion, whatever it says on a person’s business card. What has always made me immensely proud to be a part of this industry is the mutual respect and admiration that most – in fact, I naively thought all until now – analysts have for each other. We may play for different teams; we may not always like each other. But we have all achieved our places through merit and we all have a viewpoint that at least deserves consideration.
Simon has done nothing more than identify a niche in an already multi-faceted market that few – if any – other research houses were targeting; and he has created a small and very focused community of like-minded individuals to exploit that niche. The Skills Connection (TSC) has no desire to compete with Gartner, or indeed HFS Research. As Jonathan eloquently stated, TSC exists purely to help motivate vendors to present the strongest argument they can – delivered always through their own perspective. There is no intention to help vendors deceive a Gartner or a Forrester; on the contrary, one of the mainstays of TSC guidance is to promote the need to back up every claim with solid evidence.
This kind of advice should be of value to any vendor; but it provides a particular benefit to smaller vendors, or vendors entering a new market niche, who can feel intimidated and confused by the big research houses like Gartner. These vendors often have smaller budgets, more limited reach and lack the track record of more established peers. To them, just getting on the radar of a Gartner analyst can seem to be a daunting task. They see a relationship with TSC helping to create the level playing field that enables them to feel – rightly – that their voice has an equal chance of being heard.
So when you publicly attack a fellow analyst, and single them out for the sin of identifying and addressing a niche in the market that was previously untapped, you also insult every one of those vendors who have forged a relationship with TSC. You seek to demean and cheapen their market progress as “gaming the system”, and you help perpetuate the myth that Gartner analysts can be hoodwinked and manipulated more easily than those of other research houses.
In closing, The Skills Connection does not exist to help vendors cheat. It exists to help them be the best they can be.
It’s a great shame that no company yet exists to help you be the best that you could be, as there is so much that could be improved.
Hi Butler,
I like the idea of just using people’s last names – if not for the obvious ease of identification but also because I’d like people to greet me with ‘O’Donoghue’. The Irish root of my surname means “Brown Haired Warlord”. Which is much cooler than Ollie, which apparently means “shortened version of Oliver, often a nickname”.
In any case, I’m a little confused about the bulk of your reply. If anything we’ve come down heavily on Gartner’s side in this piece by pointing out the misguided principles of Levin’s (Simon’s) original blog which giddily posited on the deteriorating quality of the ‘800 pound Gorillas’ research because they were kicking out the old guard in favour of some of those youths. Analyst youths, rather than the ones my Grandmother watches from her living room window in case they steal some of the, frankly, tacky decorations that adorn her front garden. This discriminatory stance seems to be all but lost in your reply as you skilfully navigate your argument away from the original blog and, in most cases, this blog. With that in mind, I’ll speedily jet past 90% of your response which seems to be geared towards a profound sadness you have for how your former employer has been treated over the past 20 years.
The bit that I find particularly distressing is the disconnect between what’s being said here – and the marketing materials presented by The Skills Connection. For example, an entire section is dedicated to “moving the dot in the right direction” and “influencing analysts to deliver better results”. And yet in the comments here you could be forgiven for believing that TSC are just simple storytellers – perhaps leveraging the ancient oral histories of old to help technology vendors tell sagas that both entertain and beguile the listener.
The reality is that our blog highlighted a concerning narrative pushed by Simon that hiring young people damaged quality – we called him out on it throughout this blog. We have an obligation as analysts to call things like this out – if we didn’t people might accept the discriminatory narrative as fact. We’re not afraid to have an open debate – which is why we’re approving your comments. Honestly, it seems like you all basically agree with us, either directly in the case of Jonathan, or by presenting a counter-argument that’s directly aligned with our original point but tries desperately not to be.
I am reading the comments from Jonathan and Andy – maybe we are reading different blogs. The HFS blog isn’t an attack on Gartner – although if the MQ process lacks the objectivity to overcome a vendor’s poor communication one does wonder. The quality of a product or service isn’t really dependent on the vendors ability to market it – although it is relevant to its success in the marketplace. Unless I am missing some point.
Jonathan says we don’t understand the value proposition. If your services were aimed at marketing products better then that would be relevant – but The Skills Connection website mentions helping to move the dot on the MQ and the Wave – so this seems like a fatuous point.
I would argue that a process for determining the quality of a product or service should be as objective as possible with the design of the process where the effort should be placed. So that the overall quality of the rating were high regardless of any individual working on it – to maintain consistency across all the firms ratings and to deal with people leaving. Frankly these types of ratings shouldn’t be where Gartners "best" analysts spend there time – it shouldn’t matter who attends a briefing or interviews the clients as long as they follow the process. The HFS Top 10 process focuses on ensuring we have independent references and ratings on service quality from buyers – not a rating the subjective merits of one vendors Shakespearean tale and anothers faux Wagnerian operetta.
Frankly if you guys just spent your time helping clients with the messaging to help market their products and services better as Jonathan claims; we’d have no beef. But when you claim to be able to influence the scoring on a quadrant the claim that you can do that is either
a) bullshit (and it seems like the Mr Levin was lamenting the passing of his influence) or
b) the process is bad – whereby you use either direct influence or your amazing storytelling to bamboozle the chumps at Gartner. Because they won’t see through your cunning plan.
As for us living in the nirvana where no one criticizes another analyst? Really? As Corporal Jones might say “they don’t like it up ‘em.”
Dear Butler,
I have never met Mr Levin, and probably never will. I have no personal feelings towards the guy.
Let me just clarify a few points:
1. We’re on Gartner’s side in that blog…so the first 800 words of your reply are bullshit. We advocate bringing in emerging analysts who give a fresh perspective and we – at HFS – wanted to defend this policy of Gartner’s. It is also a policy of HFS to invest in emerging talent with our graduate analyst academy, which is about to be officially announced this week. Obviously TSC doesn’t care about HFS or what we cover (and have never interacted with anyone at your firm in my life save my former AMR/Gartner alum Jonathan Yarmis), but I thought I would add this news just in case TSC ever decided to look beyond the legacy analyst firms for future enlightenment.
2. Also, your website literally uses the phrase ‘move the dot in the right direction’. It’s *your* right to make money and you found your niche. We respect you for that – clearly vendors need some help (or psychological reassurance) when it comes to their dots – and we get into many painful dialogs with them regarding our Top Ten rankings. Our admiration for your failsafe business model was made clear. Good for you, Jonathan and Simon for finding a nice little business to ply your talents and I hope our blog got you a lot of impact in the market and more vendors calling up to leverage your dot-nudging expertise. Oh, and no need to thank us, just pretend to keep hating us and writing our rather nasty comments and tweets.
3. Sure, we had a little dig at Simon Levin, but we are defending the increasing pool of younger analysts who have been slighted by Simon’s original tirade of Gartner for retiring some older folks. Not to mention Simon bemoans the exit of Sondergard, who was actually fired for inappropriate behaviour…
To clarify, we’re just outing this age bias in the analyst industry… we need succession and progression plans, not merely maintain a stud farm to maintain the same old tired practices, which most people are fed up with.
Cheers
Phil