Measuring the Value of a Blog

A Forrester Group study How To Derive Value From B2B Blogging gained quite a bit of attention last month by reporting that the number of new business-to-business had “plummeted” in 2007 vs. 2006. I haven’t read it ($379 for a 19 page document seems a bit rich). but the gist seems to have been that blogs haven’t shown much value for many businesses.

From a marketing measurement point of view, the interesting question is how you measure the value of a blog in the first place? Without giving away any trade secrets, I can safely say that the answer is, “It depends.” Specifically, it depends on the objective you’ve set for your blog.

You did set an objective, right? I mean, no one clever enough to be reading this blog would be so foolish as to set up their own blog without thinking things through.

Typical objectives might be to attract an audience, educate readers, build a community, or establish expertise. The standard advice would be to first select an objective, then to select metrics and goals that reflect progress towards that objective, and then to measure actual results against the goals.

That’s perfectly sound advice so far as it goes. But it doesn’t address the truly critical question of whether a blog is the best way to meet the objectives you’ve selected. Maybe the resources invested in a blog would be better spent on a newsletter, or paid search, or trade shows.

This leads to the question of just what resources are being invested. It’s easy to underestimate the cost of a blog because so much of the expense is in the time to write it, which seems like it’s free. But in fact, time is often the most constrained resource of all, particularly for the senior executives who are often expected to be company bloggers. Certainly an organization must ask itself how the time spent writing a blog would otherwise be used. If a more productive alternative is available, the blog should go, or at least be reassigned to someone else with (literally) nothing better to do.

I asked myself that question some time ago, with the result that I changed from my Customer Experience Matrix blog from a daily to weekly posting cycle. (Incidentally, traffic did not especially decline.) Starting the MPM Toolkit blog pushed me back to a twice-weekly schedule, but it serves strategic purposes that make the extra effort worthwhile.

Interestingly, other consultants in the marketing measurement industry—whom you have to assume are measuring their blog value carefully—have decided not to bother. Pat LaPointe of MarketingNPV dropped his blog in April, explicitly stating that he could find more effective ways to deliver his message. In a side conversation. Jim Lenskold of the Lenskold Group told me he blogs very rarely because he can reach a larger audience in other ways for the same amount of work. Jim also made the particularly astute point that the kinds of information he wants to convey doesn’t really lend itself to the “quick opinion” format of most blogs. Another consultant whose judgement I respect, Laura Patterson of VisionEdge Marketing, seems to have no blog at all.

The point is not "blogs are good" or "blogs are bad". It's that blogs are tools which must be evaluated like any other marketing investment. The actual cost of a blog is a little harder to measure than many other marketing projects because so much of the cost is implicit (time) rather than explicit (cash outlay). But that’s not a reason to avoid the measurement effort—it’s simply a caution that you must be careful to do it right.

Social Target LLC Tracks Social Media Measurement Vendors (So I Don't Have To)

So far this blog has mostly been about the general issues of marketing measurement, rather than measurements for specific media. This is largely because there is already so much detailed information available from specialists in each field that I’m unlikely to add anything useful. On the other hand, medium-specific measurements are inputs to the broader performance measurements that I’ve been writing about. So I can’t ignore them altogether.

Measurement methods for traditional media, including broadcast, print advertising and direct response, are fairly well established. This is not to say they are simple or easy, but most of the interesting activity is taking place elsewhere. Not surprisingly, much of that “elsewhere” is on the Internet.

I see three main areas of continuing innovation in marketing measurement: public relations, Web site analytics, and social media analysis. Public relations obviously predates the Internet, but it makes the list because so much of the data gathering now occurs online, and so much content analysis is now automated because online content makes this possible. Web site analytics is certainly the most mature of these sectors, but despite the huge base of experience, I think it’s fair to say that practices are still evolving rapidly. Social media is the newest sector, especially as a discipline that’s distinct from other Web activities. It too has a large base of techniques and services in place—more than a casual observer might realize—but is certainly not yet stable. (You might add mobile marketing as a fourth area, but it is just getting established--although there is probably more going on that I've heard of.)

I may yet dig into these areas in depth. For today, though, let me point you to a company I found during my preliminary research (a.k.a., Googling). This is Social Target LLC, apparently a small independent consultancy run by one Nathan Gilliatt . I’ve only poked around briefly in the site and associated blog The Net-Savvy Executive, but it impresses me as a good source of information on social media analysis. The firm also publishes a Guide to Social Media Analysis, which profiles 31 or so vendors. I wasn’t overly impressed with the sample entry that I downloaded—it was just two pages and (obviously) didn’t get into many details. But it looks like a reasonable starting point for anyone getting oriented in the field.

Content Emerges as a Common Thread in Marketing Measurement

Last week’s post marked the completion of my initial survey of marketing reporting systems. So it’s a good time to step back and assess the state of the art in general.

As you’ll have noticed if you followed the series closely, there were several vendors on my initial list who were not really focused on marketing measurement. Once they were removed, I think it’s fair to say that most of the others have built their business primarily on marketing mix modeling. These are not products that build mix models, although many of their providers are in that business. Rather, these products make mix models more useful by combining them and applying them to planning, reporting, forecasting and optimization. So that’s one trend I've observed: conversion of mix models from stand-alone analyses to part of an integrated measurement process. But most of those products were several years old, so the trend is not a new one.

A fresher trend was efforts to provide more sensitive measures of brand value drivers. Specifically, I am referring to systems that tie changes in brand value to changes in consumer attitudes and behaviors. This contrasts with traditional brand value studies, which look at consumer attitudes at a specific point in time. As I’ve noted previously, the absolute values generated by these studies are somewhat questionable. But relative changes in these values are still probably good indicators of whether things are getting better or worse. (There’s an irony here someplace—an unreliable indicator becomes useful if applied more frequently.) Of course, there’s nothing new about tracking trends in consumer attitudes or about linking those trends to marketing programs. What’s being added is the conversion of attitude changes to brand value changes. This provides a much-sought link between marketing efforts and brand value.

Changes in consumer attitudes affect more than brand value: they also impact near-term sales. This relationship is reflected in relatively new (to me) efforts to use consumer attitudes as inputs to marketing mix models. Traditionally, the main inputs to these models have been spending levels for each mix element, with only minor adjustments for program content or effectiveness. Marketers would like to analyze more detailed inputs, but few marketing programs are large or long-running enough to have a distinguishable impact on total results. Consumer attitudes, on the other hand, do have a continuous presence that can be correlated with changes in short-term sales. The trick is linking the attitudes with marketing programs.

One solution being attempted is to aggregate marketing programs according to the messages they deliver, and then assess the impact of those messages on attitudes. That is, the mix model inputs are spending against different marketing messages. This could be used to predict sales changes directly, or to predict changes in consumer attitudes which in turn predict sales results. It isn’t quite as good as measuring the direct impact of individual marketing campaigns directly, but it does give some idea of their likely relative effectiveness, and therefore how future funds are best invested.

All of these changes show movement towards understanding the connection between individual marketing decisions and ultimate business results. The common thread is content: using content to aggregate individual marketing programs; assessing the impact of content on short-term sales results; and assessing the content-driven changes in consumer attitudes on long-term brand value. Today, these attributes of content are often measured separately. But I think we can expect them to be part of a single, unified analytical process over time.

MMA Avista DSS ...and more

I originally contacted Marketing Management Analytics (MMA) to discuss Avista DSS, a hosted service that helps its clients use MMA-built mix models for planning, forecasting and optimization. But while MMA Vice President Douglas Brooks seemed happy to discuss Avista, he said the most excitement today is being generated by BrandView, a newer offering that shows the long-term impact of marketing messages on financial results.

This is important because mix modeling shows the short-term, incremental impact of marketing efforts on top of a base sales level. BrandView addresses the size of the base itself.

BrandView works by comparing the messages the company has delivered in its advertising with changes in brand measures such as consumer attitudes. It also considers media spending and market conditions. These in turn are related to actual sales results. Using at least three years of data, BrandView can estimate the impact of different messages and media expenditures on the company’s base sales level. This allows calculation of long-term return on investment, supplementing the short-term ROI generated by mix models.

In other words, BrandView lets MMA relate brand health measures to financial results—something that Brooks sees as the biggest opportunity in the marketing measurement industry. He said the company has completed two BrandView projects so far with “rave reviews.”

That’s about all I know about BrandView. Now, back to Avista.

As I mentioned, Avista is a hosted service. Beyond browser-based access to the software itself, it includes having MMA build the underlying models, update them with new data monthly or quarterly, train and help company personnel in using the system, and consult on taking advantage of the system results.

The software has the functions you would want in this sort of system. It can combine results from multiple models, which lets users capture different behaviors for different market segments such as regions or product lines. It lets users build and save a base scenario, and then test the results of changing specific components such as spending, pricing and distribution. It also lets users change assumptions for external factors such as competitive behavior and market demand, as well as new factors not built into the historically-based mix models. It provides more than 30 standard reports showing forecasted demand, estimated impact of mix components, actual vs. forecast results (with forecasts based on updated actual inputs), and return on different marketing investments. Reports can convert the media budget to Gross Ratings Points, to help guide media buyers.

The system also includes automated optimization. Users select one objective from a variety of options, such as maximum revenue for a fixed marketing budget or minimum marketing spend to reach a specified volume goal. They can also specify constraints such as maximum budget, existing media commitments, or allocations of spending over time. The system then identifies the optimal resource allocations to meet the specified conditions. Reports will compare the recommended allocations against past actuals, to highlight the changes.

Avista was released in 2005. Brooks reports it is now used by about two-thirds of MMA’s mix model clients. The system typically has ten to twenty users per company, spread among marketing, finance and research departments. Each user can be given customized reports—for example, to focus on a particular product line or region—as well as different system capabilities. Building the underlying models usually takes three to four months, depending largely on how long it takes to assemble the company-provided inputs. (Standard external inputs, such as syndicated research, are easy.) After this, it takes another month to deploy Avista itself, mostly doing quality control. Cost depends on the scope of each project, but might start at around $400,000 per year for a typical company with multiple models.

Of course, just getting Avista deployed is only the start of the process. The real challenge is getting company managers to trust and use the results. Brooks said that most firms need three to six months to build the necessary confidence. The roll-out usually proceeds in phases, starting with dashboard reports, adding what-if analyses, and only then using the outputs in official company budgets and forecasts.

Brooks said that MMA will eventually integrate BrandView with Avista. The synergy is obvious: the base demand projections created by BrandView are part of the input to the Avista mix models. This is definitely something to keep an eye on.

'Integration' Offers Effectiveness Measurement Methodology

This blog is mostly about technology. But I haven’t finished the research I had intended this week, so I’m going to point you to an interesting methodology instead. This is from a vendor called Integration, about which I know nothing except what I found on their Web site. They are cool if for no other reason than being headquartered in Cyprus. But they also appear to conduct detailed studies of the effectiveness of different marketing contacts—something they refer to as a Market ContactAudit. (Why they make that two words instead of one or three, I have no idea.)

The Web site contains a detailed evaluation of their technology by the Advertising Research Foundation. The steps are:

• define a set of brands and contacts to assess. This is based on discussions with company managers and focus groups with consumers.
• survey consumers to find identify the “clout” of each contact type (specifically, its ability to convey information, create emotional bonds, and influence attitudes and behavior), and to find which brands they associate with which contacts
• use the results to calculate ‘Brand Experience Points’ (the clout of each contact x the number of brand associations with that contact), and the ‘Brand Experience Share’ (the target brand’s share of total category Brand Experience Points)
• apply these measures to other analyses such as identifying the most influential contacts, position of the brand vs. its competitors, and spending efficiency (cost per Brand Experience Point)

Compared with some other brand valuation methodologies, the Integration approach is quite straightforward. Much of the simplicity derives from its use of consumer surveys, which avoids the work of gathering actual data on media spending, competitive products, company financials, and so on. Of course, this comes at a cost: it requires relying on the accuracy of consumer perceptions, and doesn’t factor in elements of the marketing mix that are invisible to consumers, such as distribution. Thus, it will not provide anything close to the precise of a marketing mix model. Nor does it allow calculation of a financial measure of brand value.

Still, having a “common currency” to measure the value of contacts across touchpoints is critical to making effective resource allocations. Combining the Brand Experience Points with company media spending is easy enough and gives good tactical guidance. According to the audit, the Brand Experience Share has correlated closely with market share over hundreds of projects, so the basic consumer input seems to be fairly reliable.

The ARF audit also says that Integration provides detailed materials describing the process, which can even be executed without an outside consultant. Speaking as a consulant, I'm not 100% sure I like that, but I suppose it's a good thing from the client perspective.

The Integration Web site lists “global alliances” with a number of major ad agencies and consultancies, who presumably have deployed or adapted the methodology in-house. This gives the approach additional credibility. It also seems that the Market ContactAudit is part of a larger strategy process offered by Integration, which in turn can be part of a larger business planning process. There's even some marketing management software involved which includes activity based costing and management dashboards.

In any case, It's probably best to let Integration speak for themselves. Take a look at their approach if you have a moment.

M-Factor M3 Aggregates Segment-Level Mix Models (Which Is Cooler Than It Sounds)

I spend most of my time these days thinking about business to business marketing, where performance is measured one sale at a time. Ironically, it’s much harder for business marketers to calculate their impact on sales than for marketers in the anonymous, vastly less precise realm of consumer packaged goods. The reason: predicting individual behavior is difficult in both cases, but there are so many consumers that their aggregate behavior can be modeled accurately with statistics. Packaged goods marketers may not know the name of every tree, but they have a much clearer picture of the forest.

The main tool used to measure consumer marketing results is, of course, the marketing mix model. This is built by identifying historical correlations between sales results and inputs such as media spend, trade promotions, pricing, primary demand and competitive activities. Mix models can provide powerful insights into the causes of past performance and helpful forecasts of the impact of future plans. Even though few marketing managers really understand the underlying math, the models are well enough proven to be widely accepted.

But there are limits to what a single marketing mix model can accomplish. Most markets are in fact comprised of many different segments, based on geography, customer type, product attributes, and other distinctions. Each segment will behave slightly differently, so generating the most accurate results requires a separate model for every one. This wouldn’t matter, except that marketers work at the segment level. They have separate marketing plans for each segment and track segment results. In fact, a large company will often have entirely different people responsible for different segments. You can be sure that each of them focuses on her own concerns.

Building lots of segment-level models doesn’t have to be much more expensive than building one big model. The trick is keeping the inputs and model structure the same. But managing all those models and aggregating their results does require a substantial infrastructure. This is what SAS for Marketing Mix (formerly Veridiem) was designed to do. (See my related post ) . It’s also the function of M-Factor M3.

In fact, M-Factor was originally founded in 2003 specifically to help combine marketing mix models that were created by third parties. The company’s product can do this, but the firm found that externally-built models are often poorly understood, difficult to maintain, and inconsistent with each other. In self-defense, it decided to build its own.

Today, M-Factor developed its own model-building staff and toolkit. This allows it to develop separate models for each segment in a market—sometimes hundreds or thousands of them. These can be arrayed in a multi-dimensional cube, which allows the system to easily aggregate results or drill down within different dimensions. Sharing the same structure also makes it easy to update the models with new data and to build detailed reports such as profit statements derived from model outputs.

To go at it a bit more systematically, M3 provides three main functions. The first is results analysis: calculating return on marketing investments by estimating the contribution of each input to over-all results. The second is forecasting: accepting scenarios with planned inputs, and using these to estimate future results. The third is optimization: automatically identifying the best combination of inputs to produce the desired outputs.

The results analysis accepts historical inputs from the usual sources such as Nielsen and IRI. It then produces typical marketing mix reports on the sales levels, volume drivers and return on investment. It also provides model performance reports such as model fit and error analyses. M-Factor makes a point of breaking out the model error, to help users understand the limits of model accuracy and see how well models hold up over time. The company says that its particular techniques make its models unusually robust.

Forecasting starts with a marketing plan for business inputs such as budgets and prices. These are at roughly the same level as the mix model inputs: that is, spending by category but not for specific marketing campaigns. A typical model has 15-25 such inputs. They can be entered for individual segments and then aggregated by the system, or the user can provide summary figures and let the system distribute them among segments according to user-specified rules. The system then applies these inputs to its models to generate a forecast.

Once an initial plan is entered, it serves as a base for other scenarios. M3 displays the original inputs as one column in a grid, and lets users make changes in an adjacent column. Since the models are already built, the forecast is calculated almost instantly. Results can include a full profit statement as well as the inputs and estimated sales volume.

Users can freeze one forecast to treat it as the business plan. The system can later report planned vs. actual results, or compare the original plan against a revised forecast. The system can also project results for the current calendar year by combining actuals to date with forecasts for the balance of the period. Because the forecasts are built by the individual segment models, all results can be analyzed via drill-downs or aggregated into user-defined groups. M3 provides each user with a personalized dashboard to make this easier.

Optimization is an automated version of the scenario testing process. The user specifies output constraints such as minimum revenue levels, and driver ranges such as no more than 3% price change. The actual optimization process uses a genetic algorithm that randomly tests different combinations of inputs, selects the sets with the best outcomes, makes small changes, and tests them again. It continues testing and tweaking until it stops finding improvements.

Users can also ask the system to optimize two target variables simultaneously. What the system actually does is combine them into a weighted composite, using different weights in different model runs. It plots the result of each run on a chart where the X axis represents one target variable and the Y axis represents the other. Users can then choose the balance they prefer.

Initial deployment of M3 usually takes three to four months, including the time to assemble the historical data, build the models, and provide an initial set of strategic recommendations. Pricing is comparable to conventional mix models, although it is sold as a hosted service on an annual subscription. This typically includes monthly data updates and reports, and quarterly updates of the underlying models. End-users access the system via a browser and can run reports, scenarios and optimizations at will.

CLOSE Survey Finds Marketing / Sales Integration Gaps

CLOSE (Coalition to Leverage and Optimize Sales Effectiveness) is a “peer-led community of over 3,000 sales, marketing and channel professionals” within the CMO Council. The group recently surveyed its members (mostly business-to-business marketers) about sales and marketing integration. The report is not yet officially released, but they did send me a preliminary copy which I discussed with CMO Council Executive Director Donovan Neale-May.

In my eyes, the survey results boiled down to two main points: marketing’s main job is to provide good leads, and alignment between the two groups depends more on processes than technology. Neither of these is surprising. But there were some anomalies that are worth considering.

Let’s start with the role of marketing. The survey asks about this in several ways, but the most telling question was, “What metrics and measures marketing should use to quantify its impact on sales results and business outcomes?” The top answers were unambiguous: 19% said “pipeline and prospect flow” and 18% said “volume and caliber of leads.” No other answer had more than 12% of responses. So it’s clear that marketing’s job is to get good leads, right?

Not necessarily. When asked what “role” marketing should play in optimizing sales performance, there was a statistical dead heat between lead generation (29.1%) and providing sales materials (29.2%). Effectiveness measurement followed close behind (24.5%). Those are three very different things.

In another question about how marketing is “viewed” by their organization, by far the top answer was providing content and sales materials was by far the top answer (41%). Answers relating to leads and demand generation combined for another 32%, while the remaining 27% pretty much said marketing was useless. (I’m not exaggerating: 15% chose marketing provides “no real customer insight or value-added thinking” and 12% said marketing “operates in a vacuum; programs do little to affect sales.” Ouch.)

So: leads are the main measure of marketing impact, except that producing sales materials and analysis are just as important when it comes to marketing’s role or how it is viewed. This seems like a contradiction.

Neale-May’s take was that marketing is viewed as tactical (i.e., a provider of sales materials) because it doesn’t think or act strategically. He felt that marketing would be more effective and get more respect if it took more responsibility for lead nurturing and measuring final results, rather than simply catching leads and passing them immediately to sales.

It sounds so crazy that it just might work.

Back to the survey. When asked to list the key elements to maximize sales, the number one response was “lead quality and ROI” (52%). I suppose this explains why “better integrate and align with marketing” showed up as the highest ranked way to improve sales effectiveness (41%). That is, working more closely with sales would help marketing to generate better leads.

There’s just one problem with alignment: few people seem to do it. Only 16% of the respondents reported an “extremely collaborative” relationship between marketing and sales, although another 40% shrugged that they had “relatively good information sharing”.

Even scarier, less than half (42%) reported “any” formal programs, systems or processes to align sales and marketing, and only half of these (47%) said the programs were successful. That means three-quarters of the companies are not addressing alignment effectively.

One bright spot is that respondents do seem to recognize that the key to alignment is process, not technology. At least, that’s how I interpret their citing “limited processes and systems in place” as the largest challenge to integration (41%), followed by “reporting and organizational structures” (30%) and “siloed operations” (29%). The truly technical issues of “no shared data and real-time information” rank just sixth with 20%.

In terms of existing technology, 12% reportedly live in the paradise of a “well-integrated, real-time view of all customer interactions; readily accessible on-demand by all functions.” Another 37% report that “sales has good visibility into prospects, pipeline, deal flow and conversion rates”. But the other half lives poorly indeed: 20% report that “marketing hands off leads to sales and has no insight into conversion and close process”, 13% report that “most leads are never captured, qualified or acted on”, 11% have “no customer relationship management system or on-demand CRM service in place”, 7% “still use spreadsheets for tracking targets and prospects” and 1% just plain “don’t know”.

The numbers are somewhat similar for CRM systems. A lucky 13% report that CRM is “highly valued and widely deployed” and another 42% say it is “growing acceptance and adoption”. Again, the other half are in bad shape: 15% say the system is “difficult to customize and use”, 10% report a “high level of dissatisfaction”, and 21% have “no CRM system in place.”

Analytics are a slightly different story. A near-majority (46%) report that sales and marketing can both access customer analytics, while another 8% each report that only sales or only marketing have access. This leaves a little more than one-third flying blind.

Or is it really much worse? On the specific issue of “tracking and optimizing customer lifetime value and profitability”, just 6% said they had already made a “significant investment in analytics and programs.” Half the remainder (46%) are working on it, while the other half (48%) apparently are not.

But while just 6% have significantly invested in analytics, 24% list analytics as the best way for marketing to help sales to grow customer value. That was the most popular answer. The difference between 24% and 6% suggests an embarrassingly large gap between what marketers say and what they do.

Over all, it seems that about two-thirds of the companies have reasonably good customer data and analytic tools, but a much smaller elite--fewer than 15%--take full advantage of them.

Neale-May commented that marketing often does not have full access to CRM data. But he added that many marketers could make better use of the tools they do have available. Specifically, they must track prospects through the end of the sales process to understand what makes a quality lead. And producing higher quality leads is what really counts.