After a 16-year run that began with the Malcolm Gladwell bestseller The Tipping Point, the market appears be catching up with the influencer economy. As an unnamed social media exec bemoaned recently, the teens who get hundreds of thousands of dollars to take photos “no longer value their art.” That executive predicted that brands will soon realize that paying such influencers exorbitant fees doesn’t help the brands they’re supposed to promote.

That’s not necessarily the case. Like any other form of media though, influencers are neutral. It’s how you work with them that’s important—and too often brands do so incorrectly. Consider one recent example. Instagram influencer, Keeping Up with the Kardashians’ Scott Disick, recently merely cut and pasted a UK “fit tea” brand’s social media instructions for a sponsored Instagram post and then ran that communiqué in his feed. Oops.

While that’s an extreme case, it’s the more mundane fumbles that can drag a brand down. Here’s how to work with influencers wisely:

Don’t be blinded by follower counts

People with millions of followers are known for something. If that something aligns with your brand’s mission, then that’s great. If not, the public might view it with skepticism. Microsoft learned that lesson the hard way when it made Jessica Alba, who has 9.2 million Twitter followers, a spokesperson for the Windows Phone in 2013. Alba was photographed that year holding an iPhone—the same year that another mobile brand ambassador, BlackBerry “creative director” Alicia Keys, was caught using an iPhone.

Rather than going for an influencer with a massive follower base, look for one that may have a smaller one but who knows your category and is genuinely enthusiastic about your product or service. For instance, financial guru Dave Ramsey has over 500,000 Twitter followers, which is about 45 million fewer than Kim Kardashian. If you are marketing a mutual fund, whose endorsement would you want?

Check the tone

Influence matters, but so does the tone of the influencer’s communication.

Comedian Rob Delaney, for instance, has 1.3 million followers on Twitter. He also uses strong language and employs a brand of hard-edged humor that is hilarious to some and offensive to others. (Recent tweet: “Sometimes when God closes a door, He also throws a Molotov cocktail down the chimney.”)

What kind of brand would want to align with that sort of content? Klondike, for one. The ice cream brand, looking to catch the attention of millennials, launched a social marketing campaign with Delaney in 2012, The Klondike Comedy Showcase. A more mainstream consumer packaged goods brand might not have been able to pull that off. In that case, Delaney’s tweets could be taken out of context and presented as another social media branding faux pas. Or Delaney’s outspoken liberal outlook might turn off conservatives.

This matters because when you sign with an influencer, you’re in effect saying “This guy is setting the standard for our brand.” This can be a vexing issue for brands that are throwing in their lot with dozens or hundreds of social media influencers. It’s bad enough when a pro athlete endorser lands in jail. Try keeping tabs on everything your influencers tweet, Instagram and snap.

Get a brand advocate, not an influencer

What’s the difference? The former might be described as a superfan while the latter is merely a celebrity (albeit on a small scale at times.) The key is that for this person, affinity with this particular brand is a defining feature. For marketers, a potential endorser’s passion for the brand should outweigh his or her following. In the 2015 Nielsen Global Trust in Advertising survey, only 8% of respondents in North America said celebrity endorsements resonated with them. Some 66% said they trusted consumer opinions posted online.

While these practices can help steer your influencer marketing program, beware: Even in the best case scenario, working with brand advocates has risks. However, there are clearly smart ways to minimize risks and harness influencers. If you’re lucky, you might even find some who still value their art.