The Hadoop recruiter minefield

There's a lovely post from Meebo on The Recruiter Honeypot, which looks at how recruiters work from the startup perspective.

I have no experience of that. What I do have is a fleshed out LI profile, on account of that is where I filled in my resume for the Hortonworks team, rather than have some paper thing elsewhere. Now I can see who is recruiting for Hadoop skills -and who is good and bad at it.



The Meebo article praised trying to contact recruiters by phone. That may offer a good time-to-hire strategy, but I'm not sure. A London based recruiter tried that to get me to join Spotify in Finland that way -by phoning me at work.

The conversation went something like

me: "Hello?"
them (excitedly): "Hello, I'm ... I'm phoning up to see if you're interested in working at Spotify"
(tersley) "How did you get this number?"
(smugly) "I called your front desk and asked to be put through"
(tensley) "You interrupted me"
(obliviously) "Spotify are a great startup and they've got vacancies in Finland..."
(tenseley) "I don't like being interrupted when I'm coding"
(mildly apologetically) "Sorry, but there's this great opportunity and -"
(interrupting) "Don't call me at work again"
(desperately) "But, but, -how can I contact you?"
(harshly) "email -work out the address yourself"
hangs up.

That sixty second phone call broke my concentration. It may have cost the recruiter half an hour of time to work out the phone number, dial through etc, time they billed their customer. But it also cost me half an hour of my time which I happen to value more than their time. And it left me bearing ill will towards both the recruiter and their employer.

A day later they contacted me via LinkedIn. Not via any of my email addresses -all of which are easy to find if you put some effort in- but via LI.

Following on from our conversation.

I am currently recruiting for a Technical Lead of Analysis at Spotify, based in Stockholm. Hence, I would be very keen to talk to you directly to discuss your situation and the role in further detail.

I would be very keen to speak to you ASAP to send/brief you on the role and salary package, discuss your current situation in further detail and identify whether the role would be suitable for you or anyone else you may know?

If you are interested, please do provide me with your contact details (mobile or skype id) and a preferable time to chat.

My reply essentially told them that their phone call was an unwelcome interruption and that I was not prepared to talk to them about anything.

-Not interested

-I will note that you haven't done enough research to see what bits of Hadoop I work on -which, given its open source, is pretty weak. 

-I was pretty unhappy you called me. As a developer, I can't work with interruptions, and don't like them. Calling me disrupted what I was doing and cost me time. Your tactic may work for other vacancies, but for software jobs your strategy seems designed to alienate your potential candidates. It's also why I won't provide you with details of anyone else who may be suitable: I don't want them to blame me for ruining their coding times.

I'm not going to mark you as a contact, as per http://bit.ly/m0j1qM

-Steve

To design and build great things, you do need time to get things done without interruptions. Any company that implicitly permits developers being interrupted without warning before they've even joined the company is a warning sign. It'd be like a job interviews that require three meetings with powerpoint slides and an interviewer calendar-scheduling problem that is NP-hard. That's an organisation trapped in Outlook.

I was findable by email -if the recruiter was serious they could even have worked out my home phone number. But no, they chose their standard algorithm "call them at work", despite the fact that this algorithm is likely to find their target in one of two states "doing something like emails where they may be amenable to interruption, especially if they are looking for new employment" and "busy"

I don't know the probability of developers being in either state (I prefer "busy"), but if you are recruiting you do have to consider that the "call up your targets at work" algorithm may lead to success -but it can also generate more ill will towards your company than other techniques. Oh, and when looking for developers, you want the ones who can be productive.



Looking at LI, approaches come in various forms.

1. The "friend request from a complete stranger".

Daniel P- has indicated you are a person they've done business with at F-: 

Apologies for contacting out of the blue but I was wondering if you would be interested in discussing an opportunity with a company developing real-time machine learning software. If this is not of interest please archive. Dan

I dismiss these automatically because of my "how can we be friends if I don't know you" policy, which  is clearly stated on my LI profile. People who don't read that haven't done any research at all. Why this policy? It says "do not disturb". It also pays for the salary of Wittenauer, Jakob, and others that I have respect for. Recruiters who don't actually bother to pay LI the money for the premium recruiter account aren't serious professionals, nor do they contribute indirectly to Hadoop.

2. The desperate approach from somewhere round the world. In the past month I've been contacted by some Russian company offering a choice of London, NYC or Moscow, and Japanese company.

"I came across to your Linkedin profile and wanted to speak with you about the possible job opportunity in Tokyo. My client is a global internet conglomerate based in Tokyo and they are looking for Hadoop engineers for their big data group. 100% of the daily communication is being done in English in this team so no Japanese language skill is required. If you are interested, we can have a casual discussion over the phone or Skype to talk more about this position. Please let me know how I can get in touch with you."

I view this as a metric of Hadoop's success -and a sign of the fact that companies trying to make Hadoop a central part of their business. It also could be sign of the company being in some high-visibility firefighting project that is desperately trying to upskill to meet utterly unrealistic expectations, committing on some project for which they don't have any understanding of at all. I do feel sorry for everyone involved here -and so the "sorry" note was at least polite.

3. The utterly under-researched approach. These are just painful. Here's one I was tempted to forward it to Mike Olson and say "if these are your partners, I'd worry about their data mining skills.

Take this one.

Steve,

I'm conducting a selective search to identify a knowledgeable individual that could assist G- in standing up an authorized Hadoop training practice. Your profile came to the top of my list.

G- has entered into a partnership with Cloudera.

If you could you let me know your interest in working with G- in a contract capacity to assist us building this product line, I'd certainly appreciate it. Also, if you know of anybody that may be interested in such an opportunity, please point them my way. thanks in advance for responding to my inquiry.

No attempt to consider the fact that I now work with the experts in Hadoop internals, the best QA team I've ever met, and am having lots of fun. Instead they say "would you like to work as a contractor teaching people about Hadoop" and say "we are a partner of Cloudera" as a metric of competence. Yet they are clearly not au-fait with the Hadoop world, including the fact that Hadoop experts are being offered jobs in Tokyo, Moscow and London, so there is no need to take up some contract position where your work alternates between "unpaid" and "doing the same course again".

Tip: if you are trying to get one of the core Hadoop developers to work as an instructor, you have to make it appealing. That wasn't it. I'd recommend something that not only competed with the #2 approaches in terms of compensation, but offered opportunities to do interesting Hadoop-related work as well as lecturing. Something like "three days a week you'll be teaching everything from basic Hadoop up to how to make Hadoop a key part of the next generation of enterprise applications. The remaining two days will be spent working in a part of Hadoop of your choice -including the new layers, tools to help end users, and doing papers and research in the system. We will also provide $XYZ of EC2 cluster time for you to do that work at realistic scales.

4. The very ambitious NIH complete rewrite of everything approach. While I'm not interested in working on these, they are fascinating enough that I follow up just to find out what is happening. A couple of months ago someone asked me if I was interested in

"a role working for a company who rejected Hadoop for their own in-house developed tool which can handle more data?"

This obviously merits a followup, given that known users of the stack include: Y!, Facebook, LI, eBay, Twitter -and that MS effectively stopped work on Dryad because even they couldn't justify the R&D spend for producing a complete competitor to a platform that is getting more sophisticated over time.

Your customer is clearly optimistic that they can sustain a competitive edge against the combined R&D spend of the Hadoop community, and have also taken a strategic decision to abandon the emergent layers above Hadoop MR as well as the integration tooling with data sources and back end databases.

their reply.

"They're not looking to commercially compete with Hadoop but have simply rejected it in favour of their own tool which they feel offers more to their machine learning based software used for real-time 1to1 customer intelligence across multiple touchpoints."

I can see some of the appeal there -some aspects of the MR layer is biased towards scale over latency (e.g. heartbeat-driven workload pushing out, creation of new JVMs for specific task fractions. But I would not start from scratch there, as what you gain in speed you lose in the NRE costs of developing your own stack, the fact that you don't have the tools above: Pig, Hive, Mahout, Spring Data, and the fact that while there is a shortage in Hadoop skills, you can at least be confident that there is more than one person in the world outside your own company that knows Hadoop. No internal project has that luxury, which is why the recruiters are forced into approaching the Hadoop people. Any of whom will need retraining before they can contribute or use the in-house platform. That's the problem with NIH -the stuff you build is isolated.



The approaches which do at least get a polite declination, then are:

Emails to me that know who I am. Special mention of the Facebook recruiters here who actually cited some of my public presentations and then explained how that work would related directly to their problems, and why working on that stuff at Facebook would be even more interesting, challenging and fun.

Emails to me from people I know -and introductions from others. The apache & UK development community is not that big -if you know someone in that world, they may know others. If LinkedIn says someone is one hop away, rather than doing the unpersonal "LI introduction" thing, get the contact you have in common to do an intro via email. Use the LI graph to your advantage, but personalise the approach. Example: someone from Microsoft who saw that we both knew Savas, got him to do the email intros.

One painful failure here is recruiters who didn't look at the expertise & experience of their employers, and so failed to turn anonymous approaches into direct approaches.

As an example, I got hit on by someone recruiting for IMdB who didn't look at the LI histories of the founders like Col Needham and his colleagues, and so didn't know that the original Bristol group are all ex-HPLabs.

" Would you be interested in IMDb? They would be particularly interetsed in your Hadoop and and Cassandra experience. "

I forwarded that to Col and said "get them to do their homework". Interesting to know that IMdB are playing w/ Cassandra though -I wonder if they done the Perl to ZK bridge too?

Others that spring to mind here include Google and Amazon. Yes, they are big an anonymous companies, but at least build graphs of your staff and people you are curious about.

Therefore, here's my counterpoint to the meebo story for the recruiters themselves.

1. Do your homework before contacting anyone. That's shared history with staff from the co. you are recruiting for, specific skills of the person you are approaching -whether or not that actually matches your needs, etc. If there is shared history, approach the contacts in the company to see what they know of/think of the "target," then get them to reach out direct with an introduction. You may think you lose your fee, but if the target knows people in the company you'll do that anyway.
   
   
2. Personalise. That means look at the specific skills and experience of the target and shape your email accordingly. Look at the blog postings, what they say on twitter, anything stuck on slideshare. Look any papers and books they've written -and read them. If they've done things like that it may be their skills are not what you are looking for, they may have opinions you don't want -or it may be they are even better than what you were expecting and you should invest more effort in that individual than others.
3. Make the lifestyle appealing. Personalise more than just technical issues. Look at the target's place of residence, lifestyle, etc. And shape accordingly: "our office is based within half an hour of some excellent skiing," "we have a great team who love to party out hours," "the schools in the area are good". "housing is affordable," "you can work from home 2 days a week, and at customers on the continent 3 days".
   
   
4. Make it appealing and interesting. That's not just compensation, but things that may appeal to developers in this world: scale of operations, area of work, algorithms ("cutting edge graphs", "near-real-time", "tier-1-web-property"), hardware ("ASIC-hosted algorithms with multi-TB of RAM/server). On that topic,  "an office kegerator" is not that compelling. Near my home I have the reference public houses of The Highbury Vaults and The Hillgrove within two and six minutes walk respectively, along with some tier 2 and tier 3 establishments : Cotham Porter Stores, Beerd, the White Bear, Colston Arms, the Scotchman and His Pack, the Robin Hood, the Green Man, the Hare on the Hill and the Kingsdown Vaults. My beer storage infrastructure is outsourced into a High Availability solution with both redundant service nodes and multiple routing options.
   
   
5. Don't end your missive with "do you know anyone else who might be interested?" This is the recruitment equivalent of saying "I have admired you for a long time and would like you to be my [girl|boy]friend -and if not which of your friends am I likely to get into bed with?" It destroys any perception that the recipient is considered unique and valued. Don't destroy the recruitment opportunity by trying to get the candidate to do your work for you.
   
   
6. Never phone me at work. Which means, now I work from home up until the late evenings, never phone me at all.

[photos: the abandoned netscape campus. Nothing left in the parking lot but a slowly rusting mercedes. Netscape took the web mainstream, and died in the process.]