If you haven't already done so, it might help to read this eBook on AI for Research & Analytics before you go any further. Or just read it later. Whatever. I'm not trying to tell you what to do.
"There are three of us in this relationship now"
AI, Machine Learning, automation - it doesn't matter what you call it - software is eating the world of research and analytics.
Working with huge data sets, getting projects done faster and reducing the time spent on boring stuff - these are all great benefits.
But there's another side to it, and one that doesn't get so much attention in all the talk of shiny new tools. As tech firms take a bigger share of budget and attention, the old agency / client models are straining and starting to break down.
I consult with all sides in this game - with insight teams, agencies and tech providers - and I get to see the challenges that each party faces. The reality is that the rules of engagement are changing in ways that are messy, confusing and commercially difficult.
Clients are partnering directly with tech firms; agencies are told to deliver more from a shrinking revenue base; tech firms struggle for the growth rates their investors demand.
In this article, I'm going to explore ways in which client-agency-platform relationships need to evolve as machine learning and automation transforms the world of insight.
What's going on with clients
In-house teams are under pressure.
It's not just the squeeze on headcount, budget and timescales - insight leaders are told to make radical changes but keep the plane in the air while they do it:
- hiring and training for new skills in data analytics, digital and strategy
- investing in new software tools and data sources
- giving more strategic direction to stakeholders
- proving the commercial impact of their work
- supporting 'embedded' insight models in which other teams - product management, brand, CX - do their own research with self-service tools.
And the list goes on.
The days of a research manager running process between stakeholder and agency are long gone.
And all this pressure flows downhill ... to agencies.
What's going on with agencies
Three big drivers are impacting agencies.
Some of this is legitimate - these days, agency models are too costly for certain types of work.
But some types of work just cost more - they need to be done carefully by experts - and that comes at a premium. You can't escape it. Procurement teams can rarely tell the difference between the two, so the baby gets tipped out with the bathwater and margins get sucked out of good quality work.
Today's pace is the slowest it will ever be - blah, blah, blah. So business is speeding up.
Most agencies are actually hard to make their processes leaner. But clients forget this means no time in the budget to do follow-ups or add contextual insight around the core brief. And with fewer hours to bill, the pressure on overheads goes up.
So far, this has been an overplayed benefit to clients and an over-hyped risk to agencies' existence. But we're in the midst of an evolution, and it's early days for these new models.
There's clear growth of in-house delivery of some projects using software tools. But most client teams (plenty of honourable exceptions - don't shoot me) lack the right mix and culture to capitalise.
So what happens? Agencies are expected to pick up the slack - helping new platform partners, learning new software tools, building new ways of working.
Change always hurts, especially when it feels imposed by a third party.
For agencies, this can add up to a pretty brutal business environment.
They usually have big overheads as a share of revenue - for staff (mostly graduates) and premises (mostly city centre locations).
You can't normally break a lease at short notice and you can't shed people without compensation. These high fixed costs mean constant pressure to maintain revenues.
This is what makes continuous relationships and tracking work so attractive; but it's also why many agencies will discount to the point where profit disappears. There may be no margin left - but it's still revenue, and at least the rent and salaries are covered.
In the UK, a frightening share of research agencies live very close to the edge. Check out this report based on financial analysis of the sector. It shows that a third of companies are loss making.
What's going on with platforms
The exploding world of research-tech is facing a different set of challenges altogether.
Some companies manage to bootstrap, but many take expensive funding from impatient VCs just to get their product to market.
(It always costs more and takes longer).
So yes, they seem well-financed, but the investor pressure can feel more like a curse.
Even with a strong product, adoption can be painfully slow. Good sales talent is thin on the ground in this industry.
Business models rely on high spend per customer or multi-year contracts - but clients often cancel early when they don't see the promised benefits.
And it turns out that agencies have the best people to help make clients successful with software - and they're the least inclined to do so.
At its worst, this turns into a three-way Faustian pact where all parties need each other - but also really piss each other off.
It's clearly unsustainable.
Clients need direct access to platforms so they can be more agile. They also need agencies for their expertise, flexible capacity and external perspective.
And - what everyone is slowly realising - clients often need an agency to help them make the most of their platforms.
Agency people are used to running projects using software tools; in-house teams can't magically add the skills they need into their existing headcount.
So here are some ways to get this three-way relationship working better.
5 client-agency models for the world of AI
Proper agency retainers
It still amazes that 'strategic agency partnerships' are either based on continuous tracking or a promise of repeat ad hoc projects.
It takes massive effort to get on a roster - to earn the right to pitch against other agencies for each new project.
Creative, media and digital accounts run on multi-year deals; they have embedded account teams and feel part of the client's business. Why not research? The dollar amounts are smaller, but still big enough to make sense.
Endless proposal-writing and pitching serves no-one. The research paradigm is shifting from projects to programmatic: always-on data sources, regularly updated models and integrated insights.
So find the right agencies, spec the roles and metrics and get the relationship negotiated sensibly.
It's long overdue.
Customer success teams
Client teams are taking on more software tools; some of these need continuous external help.
This help isn't just a helpdesk for tech support; but neither is it a fully-fledged research team. The ideal mix is 1 part project manager, 1 part coach, and 1 part software expert.
In software, it's called Customer Success.
Many tech platforms provide this in-house; but increasingly, as they get squeezed for margin, this is a role that agencies can take on. It needs a slightly different staff profile, and it's not 'strategic' work (whatever that is); but it maintains long term connection with a client and helps generate that retained revenue.
You can spend less and get more impact from the right senior person 5 days a month than from a full time junior hire.
Don't believe me? Do the maths.
Say an external person with 15-20 years' experience charges £600 a day all-in, 60 days a year, £36k annually.
A junior on a £30k salary really costs £50k fully loaded and is effective for 180 days after holiday, sick leave, training days, corporate do-good days, hangovers etc.
And that senior person will get up to speed faster, work solidly, engage senior stakeholders, take almost zero management time, ignore office politics and do most tasks in less than a third of the time the junior hire takes.
And I'm not even exaggerating much.
There are more and more independent researchers and micro-agencies of 2-3 people who can work like this. Plenty of good talent wants to work flexibly. In the UK, the Independent Consultants Group has doubled its membership over the last couple of years.
Of course, these longer term models don't work for everything. Work ebbs and flows, and team capacity varies with it.
Sometimes you just need work done urgently.
A handful of expert researcher marketplaces have emerged over the last few years, and I expect we'll see more of them.
Think Upwork, Elance or Feelancer.com - only with genuinely capable researchers. Post a project, get your bids, choose a researcher, get the deliverable, pay the fee.
These solutions will be part of a resourcing portfolio - they'll complement, not replace agencies and recruitment shops. But they're a good option for flexible headcount, platform or domain expertise and specialist market support.
Ad hoc agencies
OK, so this isn't new. It's the dominant model today.
But as the other models take hold - mixing long term retainers, flexible support and micro-engagements - this will become the exception rather than the rule.
There will always be a place for agencies who provide niche expertise on an ad hoc basis - ethnography, semiotics, complex analytics and premium advisory services.
And it's right that they pitch competitively, because ad hoc briefs need creative solutions.
But this shouldn't be the dominant way that clients engage agencies.
So the hard work is making this real.
Here are some brief tips.
Attack your fixed cost base. Decentralise where possible, balance your full time staff with a deeper network of independents. There are plenty of them.
Learn about and trial tech solutions ruthlessly. Not all agencies can build their own platforms as Kantar has done - but there are hundreds of ways that software makes incremental improvements to the research process.
Engage with the tech platforms and explore how you can work together. Some are actively seeking consulting and customer success partners; others have certification programmes that can help you win new business.
Build a digital insight transformation roadmap that includes in-house talent, software investment, partner relationships, workflow changes, financial models.
Include the business case for new commercial relationships with agencies and external talent.
Start to trial the researcher marketplace and flexible talent options.
Build retainer models in partnership with your strategic agencies: give and demand transparency on budgets and costs; offer 24 months' minimum commitment.
Build equitable agency partnership models. They can be far more that just a route to market - they are enablers, on-boarders, delivery and support partners - but they need the right mix of financial incentive and training to get there.
OK, I didn't say any of this was easy.
If it were, I'd be out of a job.