Code, Picasso, and Simplicity

I coded all day yesterday, focusing on a new feature for Hirelite: Speed Dating for Software Jobs. I implemented ~300 lines, added tons of functionality, and was feeling crazy productive. After this surge of productivity, though, the familiar pangs of technical debt anxiety came. I prescribed a hearty dose of refactoring.

I looked for commonalities to refactor among the eight paths of the feature. In particular, two paths stuck out as having more in common with the rest of the feature than the other six. Coincidentally, these two paths were far more commonly used than the others. Additionally, these two paths could effectively cover the scenarios implied by the other six.

Interestingly, these commonalities emerged through implementation. As I implemented the eight paths, the data structures backing each path generally fell in to two buckets. Each of the two more commonly used paths took advantage of different data structures, allowing the less common paths to be covered by one or a combination of the two.

These things weren't obvious from the user experience or product perspective. It took a deep, code-level understanding of the details of the problem and multiple potential solutions to surface this insight.

By the time I actually finished the feature, it was down to 50 lines. Though the simplified solution now seems obvious, I'm sure I would not have found this shortcut without working through the initial cases and writing the 300 lines. 

It always astounds me how much work it takes to understand something well enough to truly simplify it. Sometimes I feel bad, like I've wasted a bunch of time building something that should have been easy from the start. Or maybe I shouldn't have done it at all - I should have hired a rockstarguru who doesn't need to write rough drafts of anything, ever. And then I remember those people don't actually exist.

The same seems to be true for art. Picasso said:

... pictures went forward toward completion by stages. Every day brought something new. A picture used to be a sum of additions. In my case a picture is a sum of destructions. I do a picture - then I destroy it. In the end, though, nothing is lost: the red I took away from one place turns up somewhere else.

Picasso simplified masterfully. In 45 days, he drew 11 bulls, eventually boiling a bull down to just 9 lines.

Pablo Picasso - The Bull. Dec. 5th 1945 - Jan. 17, 1946. Image source and further reading: Pablo Picasso - Bull: a master class on abstract art

Despite all this, I still have this gut feeling that I should have known.

  • Do you run in to situations like this?
  • Any tips on how to shortcut early iterations that take so much time?
  • Is this the type of thing that more experience helps you get around or is it purely a creative process?

After we launch the feature, I'll try to follow up with a deeper dive in to the specifics of the implementation.

Thread Synchronization Issues & Romance

Who knew threads and romantic relationships had so much in common? For those of you new to threads, threads (and processes) allow computers to seemingly do multiple things at once, where each thing is a separate "thread" of execution. On a computer with a single processor, the processor spends short amounts of time executing each thread before switching to another thread to execute. On a multiprocessor system, processors execute threads simultaneously, switching between threads when there are more threads than processors.

For those of you new to romantic relationships, I don't have much advice for you other than: Don't tell your significant other that you're treating your relationship as a series of thread synchronization problems!

Also, I'm probably perpetuating some stereotypes here. Sorry, it just makes the examples easier.

Thread Synchronization Issues

Deadlock occurs when threads cannot proceed because they're waiting on each other.

Romantic relationship example: You and wife have to wake up at 6am to catch a flight. You half-wake-up at some point in the morning and think, "she'll wake me up," and go back to sleep. The problem is, now it's noon, and you've both been thinking the same thing for six hours. You missed your flight due to relationship deadlock.

Livelock occurs when threads cannot proceed because they're too busy responding to each other.

Romantic relationship example: When was the last time you heard an obnoxious couple talking on the phone? Think back to the end of their phone conversation. It probably ended like this.

1: Love you. Talk to you later.
2: Love you too. Bye.
... both wait ...
1: You hang up first.
2: No you hang up first.
1: No you...

You're witnessing relationship livelock. Neither person in the couple nor the couple as a whole can proceed because they're too busy responding to each other.

Starvation occurs when one thread is deprived of resources by greedy or mis-prioritized threads.

Romantic relationship example: You and your boyfriend share a checking account and deposit money into it on the first of the month. You routinely make small purchases every day. Your boyfriend rarely makes purchase, but when he does, he buys something big. After the first of the month, you successfully make your small purchases for a few days, but then your boyfriend buys an iPad. All your attempted purchases are now denied. You're suffering from starvation.

Race conditions occur when success depends on the order in which threads run.

Romantic relationship example: Your son wants to go bungee jumping with his friends. He knows that each parent requires that he ask both parents for permission. Using a clever turn of phrase, he realizes that he can exploit a relationship race condition to get what he wants by asking the stricter parent first.

Son (approaches strict mother): Can I go bungee jumping?
Mother: No, but ask your father.
Son (approaches lenient father): Can I go bungee jumping?
Father: Yes, but ask your mother.
Son: I already did.
Father: Great. Hope you have fun!

Thrashing occurs when threads make little or no progress due to the overhead of context switching.

Romantic relationship example: A couple tries to decide whether or not to get a pet. The argument gets heated. They keep bringing up unrelated topics. Each time a new topic comes up, they spend five minutes on it.

1: Having a dog would be so much fun!
2: You would never clean up after it.
1: What?! I clean all the time.
... five minutes later ...
1: Well at least I don't leave clothes all over the place.
2: Psh. I'm the only one that ever does the laundry. I can leave my clothes wherever I want.
... five minutes later ...
2: I don't know if I can talk about this anymore. I'm just going to go watch TV to cool down.
1: You watch TV all the time! We don't even need a pet. You spend all your time with the TV.

Every context switch gets the couple further away from where they started and from the problem they're trying to resolve. They're thrashing.

Busy waiting occurs when one thread continuously checks if it may proceed, robbing other threads of processing time.

Romantic relationship example: A couple is getting dressed for a party. The man is dressed and ready to go. The woman is nowhere near done. The man keeps interrupting the woman to ask her if she's ready yet. The man is busy waiting.

Self Promotion

If you're a developer and you like these sorts of problems, consider attending Hirelite: Speed Dating for the Hiring Process on Tuesday, April 27th in NYC where companies will be looking for great software people.

Black Hat Recruiter Tactics

Since starting Hirelite, where we get companies and software people talking directly, I've heard a lot of horror stories about working with recruiting agencies. When I hear these stories, I can't help but think of black hat vs. white hat hacking and SEO, so I call recruiters who engage in unethical practices "black hat recruiters". Black hat recruiters resort to the tactics below because they're too lazy to confront the real challenges involved in finding and matching good people and good companies. Please note that not all recruiters are bad, and some provide a lot of value, but this post is not about those recruiters. This post is about black hat recruiting where tactics range from lies to ethically gray practices to illegal activity (in approximate order of how common they are):

Posting misleading job descriptions - This is by far the most common form of abuse. Recruiters will post a job description for a legitimate position for a client, but falsify some of the information to entice candidates. For example, a recruiter will inflate the salary/compensation portion of the job description or inflate the job responsibilities while dumbing down the job requirements.

Posting bait-and-switch job descriptions - Black hat recruiters will advertise a job that does not exist or is already filled just to receive resumes from job seekers that they can contact about other job opportunities. This is very similar to a tactic that black hat apartment brokers use (mentioned in Rent Hop's comparison of headhunters and apartment brokers).

Surreptitiously modifying a job seeker's resume - Black hat recruiters often request a resume in a format they can modify. They will make modifications to job seekers resumes without telling job seekers and then give the modified resume to their clients. Modifications range from obscuring contact information so that the recruiter is always in the loop to more liberal modifications like inflating experience and skills. Nothing's worse than getting to an interview and finding out that you know COBOL from the hiring manager reading it off your resume.

Approaching other companies job seekers interview with - Recruiters often ask job seekers what other companies they are interviewing with under the guise of tailoring their search to the job seeker. Some recruiters will go as far as to ask who specifically the job seeker is in contact with. Armed with that information, a recruiter will contact the other companies and try to send competing job seekers. I've spoken to one job seeker who suspected this was happening and caught their recruiter in the act. This job seeker told the recruiter a friend's name and had the friend wait for the recruiter's call. The friend didn't have to wait long. Only 10 minutes after the initial call ended, the recruiter called the job seeker's friend. The recruiter denied everything.

Cold calling and pressuring low level employees - Black hat recruiters will call low level employees at a company and threaten termination and legal repercussions unless the employee passes the recruiter along to a hiring manager at the company.

Buying resumes from hiring companies - Black hat recruiters will give discounts to companies that will pass all the resumes for a particular position along to the recruiter. These resumes could be from other recruiters or from candidates who contacted a company directly.

Pressuring job seekers into interviews - Black hat recruiters will pressure job seekers into interviews that they don't want to go on. Sure, job seekers should stand up to them and say, "I don't want that job," but when a recruiter responds, "I'm not going to put you in front of <company> unless you go to this interview," job seekers may give in.

Promising exclusivity to job seekers - Black hat recruiters will promise a job seeker that they will not submit other job seekers for the same position as long as the job seeker agrees not to talk to any other recruiters. The recruiter then submits multiple competing job seekers for a position. If one is rejected, he tells that job seeker that the company decided there wasn't a fit and continues to send him to other companies.

Recruiting the references of a job seeker - Black hat recruiters request references from job seekers and recruit those references. Later, job seekers hear from their references that their recruiter pressured them for resumes to send to clients, sometimes for the exact job the original job seeker was up for!

Faking a relationship - Black hat recruiters will hear that Dunder Mifflin, a company they have no relationship with, is hiring. Instead of approaching Dunder Mifflin about working for them, the recruiter will solicit resumes from potential job seekers for exciting new openings at Dunder Mifflin. The recruiter will then approach Dunder Mifflin with the resumes they have. If Dunder Mifflin rejects the recruiter, the recruiter will tell the job seekers that Dunder Mifflin said there wasn't a fit for them.

Discrediting an employee's current company - Black hat recruiters will contact an employed potential candidate and tell them that their current company is in a precarious financial state and offer to find the employee another job. Black hat recruiters will even do this to employees of their own clients.

Simulating expiring offers - When a company sends an offer to a job seeker, black hat recruiters will tell the job seeker that they only have X days (where X is usually 1 or 2) to accept the offer; otherwise, it will be rescinded. This practice is a bit more rare because job seekers and companies know each others' contact information by this point, but I've heard of this happening to at least one company and one job seeker (separate events).

Sending false offer letters - Black hat recruiters will send out fake offer letters to job seekers for companies they're having trouble getting interviews for. Black hat recruiters rely on job seekers requesting to interview with the company before accepting the offer. The recruiter then arranges an interview with the company. If the company like the job seeker, the recruiter makes sure to process and negotiate the offer, sometimes issuing a "revised" offer to the job seeker. If there is not a fit for the job seeker at the company, the recruiter is no worse off than they started, and they just drop all contact with a job seeker.


If you're thinking that any of these practices might work for you, think again. Seriously. They may work in the short term, but you will do irreparable harm to your reputation, the reputation of job seekers, and the reputation of companies you represent in addition to possibly opening yourself up to legal problems.

If you're a company or a software engineer who's tired of dealing with these tactics, check out Hirelite: Speed Dating for the Hiring Process. We have another event next Tuesday.

Got any more horror stories? Leave them in the comments.

Why Networking Sucks for Introverts (and one way I'm trying to fix it for us)

Networking can really suck for introverts. I know because I'm one of them. You're probably thinking, "Of course! Introverts are shy and have trouble with social interactions." However, introversion is much more complex and encompasses an overlapping spectrum of feelings. Here's my take on it:

In general, the terms "introvert" and "extrovert" describe social preferences, not social capabilities, and it's important to remember that there's nothing wrong with tending toward one side or the other. Both have advantages and disadvantages (many that you can overcome with practice or adrenaline).

Problems that Introverts Have with Networking

These observations stem largely from the software-related meet-ups I've attended in NYC (Hackers & Founders, Hadoop, Android, etc), so they may only be applicable to technically-oriented introverts.

1. Making small talk

"The weather sure is ____." When introverts hear this, we immediately disengage. It's a struggle for us to realize that a little upfront investment in small talk can lead to a great conversation. Small talk is all about finding something to have a deeper conversation about, but often times, introverts get stuck in small talk ruts or completely blank on what to talk about, leading to awkward pauses.

2. Inducing awkward pauses

I've been a party to plenty of awkward pauses, both on the "caused" and the "affected" side. Awkward pauses happen for two reasons: struggling with turn taking or blanking on what to talk about. Blanking on what to talk about can happen because introverts have other interesting ideas we're mulling or we've run out of conversation topics. In the past, I've thought about keeping notes on conversation topics, but it's pretty weird to see someone whip out a notebook in the middle of a conversation, so I haven't done it.

3. Politely leaving conversations of no interest

If an introvert can't get out of the small talk stage or genuinely has no interest in the person they're talking to (imagine getting stuck talking to a someone from a recruitment agency that snuck in to a MySQL meetup), the conversation is over. Time to escape.

The polite introverts needlessly stick with the conversation, trying to think of a way to break it off nicely. From personal experience, these dreaded conversations can last up to half an hour. All the while, you're catching bits and pieces of interesting conversations all around you.

The less polite introverts either have "I don't care" plastered across their faces or just walk away. I have seen both. The latter is much more entertaining.

4. Having group discussions instead of 1-on-1 conversations

If there's a type of conversation that introverts love, it's 1-on-1 conversations. It's nerdy, but it's great to get into an intellectual property debate with one other person. However, at most "networking events" it's tough to get a 1-on-1 conversation. Often times you're stuck with a group.

Introverts have a lot of trouble with group conversations. We feel like we can't get a word in - other people are always talking! We feel like we have to keep up with the main conversation and all the little side conversations that keep splitting off. And a lot of times, it's hard to even join a group conversation.

Joining a group conversation is hard because everyone already involved is participating in the conversation. It's hard for them to include someone who has just popped in. It sounds strange, but I've seen people walk up to a group conversation, stand there for 10 minutes, and then walk away without ever saying anything.


What I'm doing about it

I've resolved to help fix these problems for a subset of introverts in a subset of networking situations. I want to help software engineers, who are definitely more introverted than the general population, network to find jobs and meet companies. To accomplish this goal (and others - a subject of another post), I'm introducing Hirelite.com: Speed Dating for the Hiring Process.

At a Hirelite event, software engineers will go on 5-minute "speed interviews" with companies. Making small talk and creating awkward pauses will be less of an issue because the conversations will be short and focused on how each party can help the other. Starting new conversations and politely leaving conversations of no interest will be of little concern due to the 5-minute time limit and rotation to the next conversation. And group conversations will be minimized: one company (possibly two people) speaking with one software engineer.

Hirelite is having its inaugural event on March 16th in New York City. For this event, there is only space for 20 companies and 20 software engineers, so let us know early if you would like to attend.

Email server recipe for avoiding spam filters

Setting up an email server and making it work with anti-spam protocols is one of those things that's 100x easier to do the 2nd time around. I follow these steps whenever I need to do it.

Prerequisites

  • Apache
  • Ubuntu 9.04 (these instructions should work for other versions, but that's what my target currently is)

Ingredients

  • postfix
  • dkim-filter - DKIM (DomainKeys Identified Mail - new standard)
  • dk-filter - DomainKeys (legacy protocol - needed for Yahoo, etc)
  • SPF records

Postfix Setup

First, we need to install an email server, postfix. I'm going to skimp a little on configuration here because with most EC2 AMIs I use, postfix is already installed. For more information see: Postfix setup and configuration for Ubuntu.

The most important parts are installation: 
sudo aptitude install postfix

and configuration of myhostname, mydomain, and mynetworks (which determines which hosts may send mail through the server). Here's the section from a sample configuration:

myhostname = yourdomain.comalias_maps = hash:/etc/aliasesalias_database = hash:/etc/aliasesmyorigin = /etc/mailnamemydestination = yourdomain.com, localhostrelayhost =mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128mailbox_size_limit = 0recipient_delimiter = + inet_interfaces = allmydomain = yourdomain.com

DKIM setup and configuration for Postfix and your DNS

Next we need to install and configure DKIM. This will make the server sign email headers so that users can assert, through DNS, that an email's origin has not been forged. It's safe to directly follow the instructions here: DKIM setup and configuration for Postfix and your DNS. The only catch may be with your DNS; as some DNS's (particularly GoDaddy) will not display the fully qualified TXT record name. If you enter "mail._domainkey.mydomain.com" your DNS may show the record name as "mail._domainkey". If it does, don't worry about it. It's just following an internal convention. When the DNS server receives queries for "mail._domainkey.mydomain.com", it will respond correctly.

DomainKeys setup and configuration for Postfix and your DNS

DKIM (described above) is the newer, standard protocol for signing emails; however, some mail providers are still using the older DomainKeys protocol (namely Yahoo). It's safe to directly follow the instructions here: DomainKeys setup and configuration for Postfix and your DNS, but watch out for a few repeats. You will have already completed the second part of the "Configurating DNS" section, and should use the second form of the Apache configuration in the "Startup and testing" section because you will have already placed a DKIM configuration in Apache:

milter_default_action = acceptmilter_protocol = 2smtpd_milters = inet:localhost:8891,inet:localhost:8892non_smtpd_milters = inet:localhost:8891,inet:localhost:8892

SPF setup and configuration for Postfix and your DNS

SPF (Sender Policy Framework) allows mail recipients to reject mail received by senders who DNS does not recognize as authorized senders for a domain. It's safe to directly follow the instructions here: SPF record setup and configuration for Postfix and your DNS, though you may need to create the "smtpd_recipient_restrictions" section in your postfix configuration.

Validating your installation

Throughout this process, you can emailing check-auth2@verifier.port25.com to Validating your email server configuration. You will receive an email reply to the address you sent the check request from. Also, make sure that the email address is from the domain you are targeting (it could be different based on the hostname or other postfix configuration, particularly on EC2). Here's an example command to send the email from your server:

echo "test message" | mail -a "From: you@yourdomain.com" -s test check-auth2@verifier.port25.com

Other limitations

From time to time, you may run across an email sending limit imposed by EC2 and you may need to request that the limit be removed.

Additionally, it's good to check that you are not listed on Spamhaus.