Quality Control: Sample Roasting

Our Probat P3 Electric 3-barrel sees A LOT of use!

October 7, 2014

We are officially hitting the tail end of the busiest time of year for receiving samples in our cupping lab. It’s no secret that the bulk of our coffees come from Central America and East Africa, and as such, we’ve received more ‘offer’ samples in the past few months than all of the rest of the year combined. This last year in Guatemala alone we cupped over 600 farm samples. Now, to be honest, some of that number was roasted and cupped at origin – maybe 200 total. But the bulk was evaluated right here in our lab in West Oakland. As such, it seems like an appropriate time for us to talk about our process of green coffee evaluation, and why if you’re running a roasting business, it’s important to have an evaluation program in place.

Many of you will find this information useful as a way to better understand the purchasing process on this side of the water. Much of our evaluation of coffee comes from it’s taste (no surprise here), and the sample roaster is an extremely valuable tool to present a coffee’s quality level. But also, I would argue that just as much stock is placed in the roaster, that is, the human being operating the machine, as the machine itself. There are good and bad roasting machines as well as roasting approaches, and we find that much of our home roasting audience are just as careful and attentive to roasting on their home machines as the “professional” production roasters down the street (we talk A LOT about “professional” vs enthusiast around here – and there really isn’t much of a difference, except for maybe equipment). Roasting coffee is hardly plug and play, but those who are comfortable with operating their roasting machines are poised to hit their marks consistently. And we’ve found that many home roasting machines work great as sample roasters, especially when you get to know them inside and out. And so we hope this article will appeal to both those standing behind a Probat 3 barrel as well an Air Crazy – hell, we’ve literally used both to evaluate samples.

Sample Size Me

For those who run a roastery, you’re probably well versed in the exchange of coffee samples (often unsolicited!). Whether you’re buying a sample from us, getting ‘spot’ offers from importers, or something direct from farmers, you’re receiving a pretty small amount of coffee that you will theoretically be able to extrapolate enough information regarding quality in order to make a purchasing decision. It can be tricky, especially when the sample is under 300 grams. If it is 300 grams, you generally can get two roasts out of it(on most sample roasting equipment), take a moisture reading (you need 250 grams for most machines), and give it a good visual check for defects. But if the sample is only 100 grams, well, then roasting and cupping will be the two most important tools you use in order to effectively judge your samples.

We use many tools for judging green coffee, but I’m focusing on roasting for this first article. Some of the other evaluation methods are tied to amount, whereas regardless of size, we always roast and cup all our green samples – even when there’s not quite enough for one full batch (because ultimately it’s about cup quality, right?). But even when there is enough for the standard Q grade check (350g), for the sake of efficiency I usually save a thorough visual check for only those coffees I’m considering buying. We get a lot of samples here, and so running every test on each coffee would be much more than a full-time job. But in the end, roasting always happens for every single sample that comes through our doors.

So What IS Sample Roasting?

3-barrel sample roaster, or Saturday movie night?

For those not totally familiar with sample roasting, the first question is usually “what’s the difference between sample and production roasting?”. One of the main differences is batch size. With sample roasting we’re looking at roasting somewhere in the 100 – 150 grams of green coffee, or enough to make 7 – 10 cups of coffee. With such a small bean mass, roast development happens more rapidly, and so your overall roasting benchmarks, including finish time, will be abbreviated. In general, we shoot for yellowing around 3:30, 1st Crack around 7 – 8 min, and ending the roast anywhere from 8:30 – 10 minutes. This is much shorter than most production roast profiles on multiple KG roasters.

Another difference is that with sample roasting, ‘profiling’ takes a back seat. This also has a lot to do with bean mass. Roasting 100 grams in a steel drum that’s 8” in diameter is very different than roasting 20 LBS in a Probat 12K drum roaster. Heat absorption, convection, conduction, are all greatly affected by the bean bed, the ambient air in the drum, drum material, and on and on.

It’s very difficult, if not impossible to transfer a ‘profile’ you come up with sample roasting to a large production roaster. If we isolate just one of the many factors unique to sample roasting, the small batch size, it becomes apparent that you can’t simply “scale up” a roast technique from a small machine to a large one. The thermodynamics of every aspect are different with a small charge of green coffee in a roast chamber; the turnaround time when the coffee starts to accept heat, the “rate of rise” in temperature in the yellow warming stages of the bean, the bean-to-bean convective and conductive heat transfer, etc.

The only aspect that may be the same are the environmental temperatures of the roast machine, and the set points at which water content in the bean becomes steam, and when the cell matrix of the coffee begins to fracture. What you can do though (and what we do when writing reviews) is get an idea of how a coffee will taste at various roast levels. This is especially useful once you’ve already bought a coffee and are figuring out how it’s best utilized at your shop.

So for us, the use of sample roasting changes with the different type of sample we are evaluating. First it is used to evaluate offer samples by doing our best to roast the coffee(s) to a point where we can effectively judge the quality. And then once we make a purchase, we roast small samples of the landed coffee to different roast levels in order to get a feel for how the coffee tastes at different ends of the roast spectrum.

What Makes a “Good” Sample Roaster?

A Fresh Roast is a very consistent little air roaster indeed
Roasting can be a lot like listening to music, in that if you’re used to playing records with the “loudness” button on, it’s probably best to keep it on when judging fidelity (flattening levels is usually recommended). Similarly, if you’re used to roasting and drinking coffee from a home roaster, and are able to keep roast times within a reasonable range, then it might not make sense to trade out your setup for a fancy multi barrel machine. I’m not saying that if you prefer dark roasted coffee, that’s the best measure to judge a coffee’s quality by. But rather, you can get pretty good at roasting samples using less than ideal (maybe ‘professional’ is a better adjective than ‘ideal’) roasters, and in a lot of cases, it’s enough to get a sample roasting program started. Many shops use home roasting equipment to test samples, and it’s not entirely unheard of to even use a popper!

Finding a roaster that works for you will really depend on your needs, and one major factor to consider is sample volume. How many samples do you plan on inspecting each week? If you’re roasting for a small shop, or only have a few wholesale accounts, maybe a roaster that handles one batch at a time will suffice. You’re probably not looking at too many samples at any one time and shouldn’t expect to spend too much time at the roaster. But if you have several shops, lots of wholesale customers, and perhaps most importantly an ever changing menu, you’re probably evaluating many samples and need a machine that roasts multiple samples at once (and if this is you, I’m sure you already have this). Roasting 100 samples a week on a single barrel machine would be excruciating.

Another factor to consider is the importance of the samples you’re evaluating. Of course, all samples are important when considering a coffee for your business. But how much coffee does that sample represent? It might represent 50 LBS, or a full container (40,000 LBS), two very different ends of the purchase spectrum. And while most fall somewhere in between this huge range, the point is that having a fully manual roaster, and one with a quality build, makes a lot of sense when it comes to precision roasting.

The most precise sample roasters are fully manual. That is, you’re able to control heat, airflow, and sometimes even drum speed. Heat sources are a thing of preference, but we have both electric and gas sample roasters, and while heat transfer is very different, both are more than capable of producing very consistent roasts. Our sample roasters are both 3-barrel roasters, which as I said earlier are necessary in order to handle a large amount of samples in a reasonable amount of time. But there are also manually operated, single-barrel options as well (check out our Quest, electric roaster HERE). And don’t discount home roasting machines. You can get a great home roaster for well under $1K (most are less than half) that will be more than efficient for most small shops.

Consistency is KEY

And of course, cupping is where we make the final judgement

So then, for us, the main function of the sample roaster is roast consistency – consistently developing coffees to the same level, just enough to taste as much of that coffee’s ‘potential’ without obfuscating good and bad qualities with flavors of roast. When roasting multiple samples, this becomes increasingly important as roast can be the variable that influences your purchase decision one way or the other. We regularly receive multiple samples from the same micro-region, same varietals, and using the same processing methods. In a case like this, we’re looking for minute differences that ultimately make a coffee preferable over another. This is where roast consistency is very important, as under or over development can hide or highlight certain notes. For example, under development can boost the perceived acidity of a coffee, give off a green/grassy flavor, or lend to paper/drying aspects in a coffee’s finish. So in the case of under development, each of these characteristics is directly tied to roast, and could lead to poor decision making.

Our electric Probat is a very consistent roaster, and while you can’t make sweeping changes in heat, once warmed, it’s fairly easy to maintain roasting benchmarks from one sample to the next. And one benefit to it being a three barrel is we’re able to track the roasts simultaneously, making sure each drum remains in relative sync with it’s neighbor. But there are other ways of tracking consistency than visual cues. Some folks use dataloggers, tracking roast curves and identifying inconsistencies.

From time to time, we will use a very basic manual data logging system to track roast consistency. The first part is determining a percentage of weight loss by simply weighing the batches before and after, and then dividing the pre-weight by roasted weight. I’ll also write down what time the coffee yellows, hits first crack, and is then pulled. This is partly because I want to replicate my roasts from one to the next. But the information is very valuable at the cupping table too. For instance, if we’re cupping several day lot separations from a single producer and one or more roasted samples are out of sync with the rest, we can look at these data sets to see if roast may be influencing our conclusion.

The “Value” in Evaluation

Evaluating the sample is your opportunity to judge a coffee’s potential, isolate “problem” or damaged coffees, or just plain pick the best coffee on your cupping table! So it’s important that you’re able to give each sample the fairest shake possible. The most important part of a sample assessment plan, is having one in the first place. And it’s pretty much a guarantee that sample roasting and cupping will be the tools you use the most, and the two which hold the most weight in purchasing decisions. So as long as you have a roasting machine that you’re comfortable with, can repeatedly achieve a certain roast development on, and that efficiently handles the sample volume, you’re in good shape. The last thing you want is an under utilized beast or over utilized popper! There are other evaluation methods used to measure the more “hidden” enemies of green coffee, like moisture content, water activity, etc…but I’ll save that for the next article.

The pinnacle of achievement in roasting samples is uniformity and repeatability, which are somewhat different than the goals of roasting for consumption; to produce the tastiest cup possible from a given green coffee. But all roasting shares a common thread when it comes to improving your results, that is, tasting. Beyond specific roasting techniques on sundry machines, you improve your results by checking them with a habitualness that might make friends think you have a serious case of OCD. Cross-checking sample roasts with the same coffee roasted on other machines, in particular other small roasters, leads to infinite opportunities to tweak your process and improve.

–Dan Wood