Hello from Oakland,

Since we last wrote, we’ve been working with the factory to finish off production of preordered keyboards and extra keycap sets. Our shipping partners have delivered our fourth production run (“MP4”) to customers, getting a few hundred more Model 01s out into the world. We’ve been working with the factory to coordinate delivery of MP5 keyboards, extra keycap sets, and replacement wooden enclosures. We had a long call with the factory last night. As of today, this is the latest status:

Keyboards

The factory started production of the 500 unit MP5 run on Saturday. 100 units in, they discovered that about 30% of the enclosures (from supplier ‘A’) had warped enough that they weren’t perfectly flat. On Tuesday, they sat down with factory A’s sales person, sales manager, engineering manager, and QA manager. They told our factory that the issue was due to the heat and humidity in southern China this summer. On top of that, they reported that they were seeing an approximately 50% failure rate of the 500 enclosures they’d already produced for MP6. They agreed that this was a really serious issue and that it was their responsibility to resolve it.

Until it’s sorted out, we don’t expect to be able to get our hands on more than a handful of replacement enclosures.

The upside is that between the 500 enclosures they’d made for MP5 and MP6, there appear to be enough good sets for MP5, which our factory tells us they now expect to have ready for inspection on Tuesday, June 5.

As of this moment, we believe that MP5 will fill all outstanding orders and that we will have some keyboards in stock for immediate sale in early June.

To be clear, we did NOT ship any kittens with the Model 01. If you received a cat along with your keyboard, it may have been intercepted by an unknown actor.

Keycaps

On the call, the factory also gave us some frustrating news about extra keycap sets. When they were inspecting the blank and QWERTY keycap sets to prepare them for shipping, they discovered that the paint…just sort of didn’t stay on the keycaps. They’ve sent engineers to the keycap painting facility and are digging into the root cause of the problem. When we pushed for a revised estimated delivery date, they told us that until they’ve figured out what went wrong, they can’t really commit to a delivery date. While not really what we wanted to hear, we can’t really fault them for that.

As we have further updates, we’ll share them here.

Firmware

Jesse and algernon (but mostly algernon) have been working on a fairly major firmware update, reworking the plugin system to be faster and more extensible. Algernon has been working on issues that prevented the keyboard from working in BIOS or macOS’ FileVault mode. Jesse’s nearly done with a major update to the keyscanner firmware, which we believe will dramatically reduce the chance of keychatter and reduce (our already pretty low) keypress latency. When everything is ready to go, we’ll push out a new stable keyboard firmware version, along with straightforward instructions about how to update your keyboard without needing to compile anything or edit any text files.

If you’re interested in what other folks are doing with their Model 01s, there’s lots going on over at community.keyboard.io.

If you have trouble with your keyboard, or have questions about your order, emailing help@keyboard.io is your best bet.

If you just want to chat, you can usually find us on Twitter.

<3

j+k

TL;DR: Kickstarter units have almost all shipped; most post-Kickstarter pre-orders have shipped; extra keycap sets are expected to ship sometime in April

Hello from Oakland,

When we last wrote a public update on January 25, we told you that assembly of MP2 keyboards had been completed and that assembly of MP3 was in progress.

Our factory was due to close for Chinese New Year on February 8. We were concerned that, if anything at all went wrong, no MP2 keyboards would ship out until the end of February, so Jesse got on a plane to Hong Kong for a last-minute trip to help the factory work through any issues.

In the end, the factory shipped out about 1000 keyboards as part of MP2 and about 700 keyboards as part of MP3. Originally, they had planned to ship 1000 keyboards as part of MP3, but they ran out of time. More accurately, they ran out of workers—by the time Jesse left China on the evening of February 6, most of the factory’s assembly and QC workers had already gone home. Three hundred keyboards were left partially assembled and waiting for testing. Those keyboards will ship as part of MP4.

As of early March, we have shipped keyboards to every Kickstarter backer who had completed their survey as of March 1, with the exception of “Mahogany Limited Edition” backers and our one customized ten-pack backer. (We’re still pushing on our wood suppliers to get the mahogany enclosures made, but are starting to explore other options.) We’ve also filled the majority of post-Kickstarter orders.

As of now, we’re within 600 orders of being caught up with preorders.

We’re currently currently expecting the MP4 production run to be ready for our third-party quality control agency to check on April 12. This is a bit later than we’d expected, but the factory has told us that they’re doing some work to improve circuit-board assembly and would rather slip a few days than ship a few more defective keyboards. Right now, MP4 is estimated to be about 500 keyboards. When we know exactly how many keyboards we’re getting, we’ll be in touch with everyone getting an MP4 keyboard to reconfirm your address. The factory has told us to expect MP5 before the end of April.

Keycap sets

The factory has told us that the injection molding shop responsible for the keycaps have been dragging their feet on the replacement keycap sets, claiming that they have a significant backlog of orders from before Chinese New Year. They’ve been telling us that they promise to have the extra keycap sets ready to send to the painting shop by the end of March. From there, they’ll be laser engraved and packaged. Once the factory starts packaging the keycap sets, we’ll be in touch with everyone expecting keycap sets to reconfirm your addresses.

Right now, we’re expecting to ship out keycap sets by postal mail, either from Hong Kong or directly from China. (Sending them by an express courier would likely cost as much as, if not far more than, the keycaps.)

Spotlight: user-created keycaps

One intrepid user, Stefan Eichberger, has actually designed and 3D printed two of his own custom keycaps for the Model 01. One of his keycaps is functional. The other is just fun.

His first keycap was a modified function key with a custom cutout, designed to be a little bit more comfortable for how he types on the Model 01.

Stefan’s second key was an updated Prog key, featuring a very important productivity tool that the Model 01, as shipped, was seriously lacking.

You can read more about Stefan’s work here: https://community.keyboard.io/t/3d-printing-keycaps/1151/2

If you’re thinking of crafting custom keyshapes for your Model 01, please do heed Stefan’s warnings about damaging your keyswitches. It’s very, very easy to destroy an ALPS-style keyswitch by snapping off a 3D-printed keycap’s stem inside the switch’s slider.

Update on wood suppliers

MP2 keyboards shipped with a mix of the best enclosures from the original wood Supplier and enclosures from Supplier C. MP3 quiet-click keyboards shipped with enclosures from Supplier A. MP3 loud-click keyboards shipped with enclosures from Supplier A.

When Supplier C’s first 500 enclosures arrived at our factory in the middle of December, the factory discovered that the supplier had not sized the cutouts for the USB C jacks correctly, so sent the parts back to be reworked. In early January, Supplier C delivered the second 500 sets they owed us. Later in January, they redelivered the first 500. Between MP2 and MP3, we shipped about 600 keyboards with enclosures from Factory C. There are about 200 more known-good enclosures from Supplier C in the factory’s warehouse, which will be used for MP4.

Sitting in our factory’s storage, there are currently 191 sets from Supplier C which we believe do not meet our quality standards. 60% of them appear to have warped slightly. 30% of them have significant discolorations. 20% of them still have USB C cutouts that aren’t sized correctly. Over the course of the past month, we’ve been negotiating with Supplier C to try to get them to rework or replace the defective enclosures.

Supplier A delivered their 1000 sets a little bit behind Supplier C, which is why their enclosures didn’t ship until MP3. As the factory started to assemble keyboards with their enclosures at the beginning of February, they discovered that nearly half of them had significant amounts of glue inside the screw holes used to attach the enclosures to the baseplates. After a quick meeting where we talked through mitigations and how to rework the defective parts, Supplier A picked them up from the factory and worked over the weekend to repair as many as they could. The next week, they delivered 300 ‘good’ sets back to our factory. The remaining 200, they deemed unsalvageable and promised to replace as quickly as possible after Chinese New Year. In the end, those 200 replacement sets were delivered to our factory at the beginning of this week.

The original wood supplier has, we’ve been told, completed 300 more sets to mostly fulfill their original obligation to us. However, our factory reports that the quality of these enclosures matches what they’ve shipped us in the past and that we shouldn’t expect to see more than a very small number that pass inspection. While Jesse was in China, he finally got his hands on a “dark-stained” wooden enclosure from the original supplier. The coloring looked ok, but when we looked carefully, we realized that the supplier hadn’t stained the parts like we talked about. Instead, they’d airbrushed them with a red-brown paint. That, as they say, was the end of that.

Update on manufacturing defects

In the last update, we wrote about the various issues with MP1 keyboards that customers had reported to us. We don’t have any significant new issues to report on that front, though the mix of issues has changed somewhat. As you might expect, we’ve seen fewer wood issues with MP2 and MP3 keyboards. At the same time, we saw more keyboards from the MP2 and MP3 batches non-functional on arrival than we had during MP1. We believe that there are a few potential causes. First, workers were likely a little bit more rushed in the leadup to the holiday and may not have been as diligent as we’d have liked. Second, at least in the case of MP2, surface-mount assembly of the circuit boards was completed along with MP1, but wave soldering was done months later. There is a chance that the long delay before running through the second oven could have damaged some chips. Going forward, they will make sure they tightly control the time between surface mount assembly and wave soldering. Third, we found out after the fact that the factory had not put every MP2 and MP3 keyboard through a 72 hour aging test as they had for MP1. For MP2 and MP3, they only tested a small percentage of the production runs. They’ve promised that, going forward, every keyboard will be run through a full 72 hour aging test.

We previously wrote about 'warped’ Model 01 stands. At the time, we believed the issue was caused by plastic parts that came out of the injection molds slightly deformed. Since that time, we’ve discovered that in almost all cases, the problem is simply that the factory’s assembly line workers overtightened the screws inside the stands. We’ve developed an at-home procedure for repairing a defective stand. You can read about it here: https://community.keyboard.io/t/a-possible-fix-for-wobbly-stands/1143/8

We’ve previously written about keychatter that some customers have seen with MP1 and MP2 quiet-click keyboards. Starting with MP3, all quiet-click keyboards have shipped with new Matias switches lubricated with a dry teflon instead of the old jelly-like grease that had a tendency to migrate to the electrical contacts.

We’ve been working with MP1 and MP2 customers whose keyboards develop keychatter (the form of either repeating keystrokes or missed keystrokes) to repair or replace the defective switches. We’ve also been experimenting with a software solution that changes how we read and process the electrical contact data from the keyswitches. We’ve got some more work we want to do on that front, but so far, it does seem to greatly reduce keychatter problems.

Spotlight: Captain’s chair

We’ve been waiting for this one.

Chris Vincent (no relation) posted about his custom Model 01 setup earlier this week. It’s awesome.

Chris built custom mounts to connect his Model 01 and an Apple Magic Trackpad to his desk chair. He spent time running extension cables overhead to keep them out of the way while he works, and added articulating arms to allow him to adjust the position of the Model 01 on the fly.

You can read more about his setup here: https://community.keyboard.io/t/my-captains-chair/1297/4

We love to see how people are using the Model 01 and making it their own. If you’ve done something neat or have ideas for a Model 01-related project, please post about it at http://community.keyboard.io

Shipping MP2 and MP3

We shipped PVT and MP1 keyboards from California. Due to a number of delays and expensive shipping errors, the experience was… not as smooth as we would have hoped, so we resolved to see if we could do better for MP2. Overall, we like the new shipping partner a lot better, but there are some things we’re hoping to improve for MP4.

When he landed in Hong Kong in late January, Jesse visited one potential new partner’s warehouse and liked what he saw. It took a few days to get the paperwork sorted out and a week later, 1000 keyboards were loaded on a truck in Shenzhen for a quick journey across the border to Hong Kong.

One of the advantages of our new shipping partner is that they allow us to choose between more than a dozen shipping options on a per-package basis. Most of our customers are in the US, so we ended up with a special hybrid solution for keyboards destined for the States. Each keyboard was individually labeled for delivery in Hong Kong, then all the US orders were consolidated onto pallets and shipped by air to a warehouse near Chicago, where they were handed off to DHL, who brought each keyboard to a Post Office near its eventual destination and handed it off to USPS for last-mile delivery. On the whole, this went ok. Not amazing, but ok. We ran into a couple of problems. First, we sent you shipping numbers too early. In the past, we’ve sent out shipping numbers only after keyboards had been handed off to FedEx. In this case, we sent you shipping numbers before your keyboards left Hong Kong. Our shipping partner’s package tracking page didn’t take into account the time needed to get your keyboards from a warehouse in Hong Kong to a warehouse in Chicago which set an… unreasonable expectation of when your keyboard might arrive. On top of that, MP3 keyboards were handed off to the international carrier just before Hong Kong had their Chinese New Year holiday, so boxes sat in Hong Kong for longer than we expected. Once packages got to DHL, a small number of them took an unreasonably long time to get to USPS. The worst problem only affected about 4-5 people who had mistyped their zipcodes. Due to a design flaw in our shipping partner’s software, city names that didn’t match the entered zipcodes were silently “corrected” as shipping labels were printed, leading to packages being marked as undeliverable and being “returned to sender.” We’re still trying to find four of them.

For the rest of the world, shipments went out by a mix of UPS, DHL, Aramex, DPEX, SF Express, and, in a very few cases, HK Post.

Many of the MP3 UPS shipments took far longer than we’d been promised. As it turned out, UPS was dramatically over capacity around Chinese New Year and we drew the short straw. More than 100 keyboards sat in a warehouse, unable to enjoy the New Year’s festivities. UPS shipments to Europe also suffered from confusing tracking updates—UPS now offers electronic customs preclearance. This means that package information is sent in advance of the actual package. This is great because it can save multiple days of transit delays due to customs agencies. This is…less than great…because UPS reports the customs-related tracking updates as having happened in the destination country, which leads to an update history that suggests that a package got to you and was then sent back to Hong Kong to begin its journey again.

A number of shipments to France, the Netherlands and the UK went by Aramex. We’re sorry. We didn’t know. They won’t be getting any more of our European business. They charged you extremely high handling fees, failed to contact you to arrange delivery, claimed they had attempted delivery when they hadn’t, and have still failed to figure out that they’ve finally delivered all the packages we entrusted to them.

Shipping and fulfillment is one place where first-time creators can really blow their budget. While the ability to ship just about anything to ato just about anywhere on earth in undoubtedly one of the wonders of modern life, the costs can be staggering. So far, despite getting huge discounts from our shipping partners, we’ve spent over $115,000 on shipping alone. This averages out to a cost of a bit more than $40 a keyboard.

Once pre-orders have been filled, we intend to ship most of our inventory by ocean freight to a U.S. warehouse, and then fulfill U.S. orders from there. International orders will probably still ship from Hong Kong. This should lower our total cost of shipping a little bit, but most importantly it will give us the capability to get keyboards to U.S. customers a few days after they order.

What’s next

We’re still pretty focused on getting the rest of your Model 01s (and your keycaps) shipped and helping you resolve any issues you find as you get up and running, but we’re starting to think a bit more about what we’re going to do next.

Right now, we’re leaning toward something more portable, a little bit more approachable and just a tiny bit easier to manufacture. We’re also contemplating a few accessories for the Model 01. As we figure out what we’re doing, you can rest assured that we’ll tell you all about it ;)

<3 Jesse + Kaia

TL;DR: MP2 keyboards are assembled and almost ready to ship out. (This should cover all remaining regular Kickstarter backers.) MP3 assembly has started. Jesse is on his way to China to expedite things. We’re working to improve the shipping experience going forward.

Hello from Oakland,

When we last wrote in December, we told you that because the new wood suppliers hadn’t hit the dates they’d promised, remaining orders wouldn’t ship out until after Christmas. Both new wood suppliers have delivered and the factory has completed assembly of more than 1000 keyboards. They’re working on the next 1000 now.

Wooden enclosures

The two new wood suppliers (Supplier A and Supplier C) each delivered 1000 enclosures in January. The first 500 delivered by Supplier C needed to be reworked to add a bit more space around the USB port. The old supplier also delivered about 500 enclosures that passed our factory’s new stricter QC checks.

As of now, the old supplier has delivered a total of about 1500 enclosures that passed QC. Supplier A has delivered 1000 that (we believe) have passed QC. Supplier C has delivered 489 that have passed QC (and 11 that were rejected) and delivered another 511 that we expect to pass QC later this week.

MP2 (Mass Production run #2)

Final assembly of 1080 keyboards, which should be enough to fill the orders for all regular Kickstarter backers, was completed this Monday. Our third-party QC agent visited on Tuesday to do spot checks. Typically, a QC agent will check some percentage of a production run. They’ll randomly select keyboards and evaluate them according to the quality standard. While they’ll record ‘minor’ and 'major’ issues, what they’re really looking for are showstoppers, what are called 'critical’ issues. These are issues that make the product unusable or might cause a customer to return their keyboard for a replacement. If no (or very few) critical issues are found, the whole run is deemed to have passed. If enough critical issues are found, the order is rejected and the factory has to recheck every single unit.

(Above: a keyboard that the QC agent found to be a bit wobbly and rejected.)

The QC agent checked 80 keyboards. In general, the QC agent said that the quality seemed higher than from the first mass production run. They found relatively few issues, but the defect rate for the wooden enclosures supplied by the old wood supplier was just high enough that we asked the factory to 'rework’ them, checking to make sure that USB ports weren’t being pushed out of alignment, and that every keyboard lay flat when checked on a sheet of glass.

The factory began that rework yesterday. They’ve been sending us photos throughout the day and we’re pretty happy with what we’re seeing. They said they weren’t 100% sure they’d finish today, but that they expect to finish today or tomorrow.

MP3 (Mass Production run #3)

At the same time, the factory has begun assembly of the next 1000 keyboards. These keyboards will fill most, but not all, post-Kickstarter preorders. Initially, the factory thought that this production run would be done by Tuesday, January 30. Yesterday, they told us that the schedule has slipped a bit and that they expect the run to be ready for final inspection on Monday, February 5.

Ordinarily, we wouldn’t be too worried about a one-week slip. But February is a particularly fraught time because of the Chinese New Year holiday. Like just about every factory in China, our factory will be closing for a few weeks. Chinese New Year is a time when most people go home to visit their families. Historically, it’s also when labor turnover is highest. So for factories, it’s a one-two punch: production is shut down for a few weeks, and when you start up again you might find yourself training up a number of new staff.

For us, this means that if we were to discover serious issues when inspecting the order on February 5, there wouldn’t be enough time (or staff) to resolve those problems before the factory closed on February 7. To minimize the chance of something going off the rails, the factory suggested we station a QC agent on their lines to supervise production and to inspect the keyboards as they come off the assembly line.

Well, we thought about it for about ten minutes before deciding that the right thing to do was to send Jesse back to Shenzhen and put him to work at the factory. So, Jesse’s getting on a plane late Saturday night and will be in Shenzhen until the factory closes their doors on February 7. He’ll be providing on-the-ground updates at https://twitter.com/keyboardio.

Shipping

If you’re a Kickstarter backer and haven’t yet received the keyboards you’re expecting, you should see email from us / BackerKit today or tomorrow asking you to double-confirm your shipping address sometime in the next day or two.

If you’re one of the first 950 or so folks who preordered from https://shop.keyboard.io, you should get a similar email over the weekend.

We’re looking at changing up how we do fulfillment. When we did the first mass production run this fall, we shipped everything to a US warehouse by air freight before sending it onward to you by FedEx Ground or FedEx International Economy. For U.S. customers this worked ok, but wasn’t terribly efficient. For customers outside the U.S., this quickly gets pretty expensive, both for us and for you. We heard from many folks based outside the U.S. that they’d strongly prefer a solution where the local postal service is responsible for customs clearance and delivery. We think we have something that’s going to work better for you.

When Jesse gets off the plane on Monday, he’s headed straight to a fulfillment operation we’ve been talking to in Hong Kong to double check that we’re a good fit. There’s also a particular Hong Kong postal service shipping option we’re hoping to use, though our package may be a couple of grams too heavy. If our package is deemed light enough, the total time-to-delivery for MP2 keyboards will be about the same as it was for MP1 keyboards, but the experience will be, we hope, much smoother. If we don’t squeak by, the experience will be similar to MP1, but delivery should be much speedier. And, of course, if the new fulfillment partner doesn’t work out, the experience will end up identical to MP1.

Keyswitches for MP2

(Above: a worker checks the keyswitches on an MP2 keyboard.)

As we wrote in the last backer update, a small percentage of folks with keyboards from MP1 have run into issues with 'key chatter’ due to an issue with how the internal lubricant was applied to some of those switches. (Key chatter is when pressing a key once leads to multiple keystrokes registering, so you end up typing words “liiike thiis”.) Most folks have been able to resolve the chatter issues at home following simple instructions we’ve sent them. To date, we’ve had to replace 0.5% of shipped keyboards for chatter issues that customers weren’t able to resolve on their own.

For MP2, we modified our hardware test program to detect key chatter on the assembly line. This won’t catch all chatter issues, but should catch many of them. At the same time, we’ve also been working on a software fix (which we’ll talk about in detail in a future update) that we believe will completely eliminate this issue in almost all cases.

Our key switch manufacturer has also been at work creating a version of the switches with a different lubricant such that the chatter issue oughtn’t come up at all. While we would have preferred to use newly-manufactured replacement keyswitches for MP2, the switch factory wasn’t able to get them made in time.

Because we’ve been able to resolve most instances of this issue without the hassle of a hardware swap, we ended up deciding to let the factory manufacture the quiet-click keyboards from the MP2 run with the same batch of keyswitches used for MP1. If you run into a key chatter issue, we’ll work with you to resolve it under the terms of your warranty.

If you’d rather wait for a keyboard with the new version of the keyswitches, we’d be happy to ship you a keyboard from what we’re told will be a mid-March production run. Just drop us a note at help@keyboard.io.

Replacement parts

We’ve promised some of you replacement stands or enclosures for issues you’ve found with your MP1 keyboards. In both cases, we want to make sure that the issues have been resolved for MP2 before we ask for extras. Once we have the replacements, we’ll be in touch.

Special edition keyboards

About 12 of you (hi mom!) backed us for 'Limited Edition’ Mahogany keyboards. One of the wood suppliers in Shenzhen is currently making us samples of the Mahogany enclosure. Right now, it’s a toss-up whether they’ll be able to get it to us before they leave for the holiday. When we’ve confirmed the sample with them, we’ll contact you directly with the details.

Extra keycap sets

The factory told us that they expect to be able to ship us the full production run of extra keycap sets in March. Even though we’d already signed off on the current versions, they’ve been pushing hard to improve the quality of injection molding and painting for the keycaps. (More on that in a future update.)

<3 j+k

TL;DR: We’ve shipped 1000ish keyboards to backers; we expect to ship the rest in Q4, despite some supply chain issues.

Hello from Oakland!

It’s been about five weeks since we last wrote. To say that the past month has been eventful would be a bit of an understatement. As of this week, we’ve now shipped about 1000 keyboards to Kickstarter backers. (If we shipped you a keyboard from MP1, you should have received a shipping notification from us last week.) We’re working with the factory to get the rest of your keyboards built and shipped as quickly as possible.

On that note: If you’re a customer and have moved since July, please update your address in BackerKit (or email us at team@keyboard.io and we can update it for you)!

Shipping and fulfillment

To date, we’ve shipped out just over 1000 keyboards to backers. For a variety of reasons, we ended up not being able to fulfill keyboards in strict backer number order. This time around, the factory sent us 200 loud-click keyboards and 802 quiet-click keyboards. This was a little heavier on the ‘loud’ keyboards than we’d expected. At the same time, many of you backed us for a pair of keyboards. We decided that it didn’t feel right for some of you to get two keyboards from the first mass production run, while far more of you got none. So, we decided that anybody who backed us on Kickstarter for a pair would get a single keyboard from the first run. Nobody will get charged any extra shipping fees due to the split shipments.

If you were a Kickstarter backer who pledged for two keyboards and chose to get both a loud and a quiet keyboard, we randomly selected which one you’d get from this run. (Fun fact: We had to run the random sorter about a dozen times before we got numbers of loud and quiet keyboards that matched the available inventory.)

As of now, we believe we’ve sent one keyboard to:

• Anybody who backed us before midnight US/Eastern on the first day of the Kickstarter campaign
• Anybody who backed us for a two-pack on Kickstarter
• Anybody who backed us for a single loud-click keyboard on Kickstarter
• Anybody who volunteered to help us test out the 'PVT’ (pilot run) keyboards

We’re working hard to get the rest of your keyboards produced and shipped as quickly as we can.

Visiting China

In the last update, we told you that Jesse was on his way to China to train up a new Quality Control contractor, since the previous one told us that he wasn’t going to be able to spare any time for us.

The new contractor has been working out great so far. His process is a whole lot more rigorous than the old vendor. Jesse spent a full day with him working through 5% of the first 1000 units one by one, building a new quality standard. After that, he worked directly with our factory’s QC team to re-audit all 1000 keyboards fixing a number of issues, mostly related to the quality of the wooden enclosures. At the end of each day, we got a report about what he’d found and worked together to figure out where to draw the line on “marginal” items.

One thing we discovered during the first day of QC testing was that the RJ45 cable vendor had made every cable 1cm longer than we’d specified, which made the keyboard a bit harder to put together than it ought to be. In the end, we decided to accept these cables for the first 1000 keyboards, but to require that they be fixed for future production runs.

There were also, still, a couple of small infelicities in the laser engraving for the MP1 keyboards. Our team from the factory took Jesse to the new laser engraving supplier’s workshop and we spent a couple hours moving individual key labels by a millimeter or so until things looked the way we wanted them to. Overall, we really like the new laser engraving supplier. The engineers working there seem to take great pride in their work and seem to really know what they’re doing. Changes that took an hour at the last supplier took minutes for these folks to sort out.

Our wood supplier

At some point, one of us remarked to someone that the wooden enclosures for the Model 01 had turned out to be less heartache than we’d expected. Well, we spoke a bit too soon.

The wood supplier we’ve been working with for more than two years has been having a really hard time meeting their commitments to us. The defect rates we’ve been seeing for the wooden enclosures have been really, really high. The factory has rejected between 50 and 80 percent of all enclosures delivered by the wood factory.

(We wish it weren’t so, but we need to leave some detail out of this story for business reasons.)

Some of the issues are related to the wood itself. We don’t allow knots with wood filler on visible surfaces of the keyboard. Similarly, “weird looking” discolorations on visible surfaces are something we’ve been pretty clear we can’t ship to you. This is the sort of thing that is a straightforward consequence of using natural materials; both we and the factory expected a certain reject rate. That hasn’t stopped the wood CNC factory from shipping those over.

The bigger quality issues are related to how the wood factory has been doing the 'finish’ work on the enclosures. The factory found a large number of enclosures where the nice curved edge of the palmrest had been sanded to an angle. Worse, a bunch of enclosures looked like someone had dremeled the cutout around the thumb keys, leaving it pitted and uneven.

Once we’d come up with the new internal quality standard, the factory’s QC went through and pulled out keyboards with enclosures exhibiting these issues. A few that Jesse personally thought weren’t very noticeable got left in; otherwise we wouldn’t have been able to ship even the 1000 keyboards we did ship. So far, we’ve had one backer report of something that should not have passed QC making it into the wild. (We’ll be shipping that backer a replacement enclosure.)

It was pretty clear that there was a disconnect between what the wood supplier thought was acceptable quality and what we were willing to accept, so Jesse, along with a team from the factory, paid them a visit.

We spent the better part of the day with the factory boss and their sales guy working through every class of defect we’d found, trying to determine the cause of the issue and an acceptable mitigation or resolution.

When we asked why the production enclosures didn’t match the 'golden sample’ we signed off on last year, the sales guy told us that they were produced using a completely different process, by a different team, on different machines.

We asked them to walk us through the production process. That’s when we found out the first 'fascinating’ detail. They haven’t been CNC machining both sides of our enclosure. They’ve only been CNCing the bottoms. Then, they’ve been applying the fillet (rounded edge) on the outside edge of the keyboard on a router table. After that, the enclosures get sent down the street to their finishing workshop.

At the finishing workshop, we found out why some of the thumb key areas looked like they’d been dremeled by hand.

The factory had been dremeling them by hand.

After that, we learned why it seemed like the fillet on the top of the keyboards sometimes ended sharply, almost like the tops of the enclosures had been sanded down too aggressively.

After the production tour, we returned to the wood shop’s business office to talk through what we could do to reduce the defect rate and get the schedule back on track.

We proposed that they could rework enclosures that had over-sanded fillets by simply increasing the fillets on those units, so long as the two sides of a keyboard match. (After all, with the over-aggressive sanding, we’d already been shipping fillets that didn’t match the design we sent them.) This doesn’t impact the sturdiness of the keyboard and, if anything, a more aggressive fillet will be a little more comfortable.

For the keyboards with discolored wood, we proposed that they try a dark 'walnut’ stain and that we would be willing to buy some quantity of those pieces from them, though we couldn’t ship them to any of our pre-order customers as a “surprise”.

For enclosures that were simply scratched, we suggested that they just refinish them.

The only things we said we absolutely couldn’t accept under any circumstance were enclosures with visible wood filler or over-aggressive dremeling.

When we left, the boss of the factory had promised to get the second 1000 enclosures to the factory before the National Day holiday started on October 1 and to get the next 1000 enclosures to the factory by the middle of October. They promised to start milling new enclosures for the third batch of 1000 on October 4.

The proposed schedule

Before leaving China, Jesse discussed delivery dates with our factory. They made us promise not to share those dates with you unless we included a disclaimer along the lines of “This is the best possible scenario. If a supplier does not deliver on time or some other problem comes up, we will not be able to meet these dates.” In literature, they call that “foreshadowing.”

If everything had gone to plan, the second 1000 keyboards would have been ready for us by October 18 and the third 1000 should have been ready… now.

What actually happened

Everything did not go to plan.

On October 4, about 500 sets of enclosures showed up at our factory. The factory’s QC team audited them, rejecting about half of them out of hand. As far as they could tell, they hadn’t been reworked at all. About a week later, this happened again.

So we thought, “Hey, we’ve got 400+ good enclosures. That’s something.”

The wood shop promised to rework the rest and deliver them to our factory, along with an engineer to work with our factory’s QC team about a week later.

Once again, they showed up with only a couple of hundred enclosures, most of which didn’t meet the quality standard they’d agreed to, in writing, with our factory. When, at the same time, they reviewed the “good” enclosures in our factory’s warehouse, they found that about half of them had unshippable defects.

For those keeping score, this was now about when we’d been promised the third batch of 1000 enclosures. The factory sent someone out to the wood CNC factory’s workshop to check on those new enclosures. It turns out there weren’t any.

When pressed, the wood CNC shop told our factory that they’d changed their mind and that, due to the high defect rate, they wouldn’t be taking the contract for the remaining 3000 enclosures, including 1000 of the units that were now overdue.

They did say that while they were committed to delivering the second 1000 enclosures, they didn’t have a date on which they thought they would deliver them, just yet.

Our factory spent some time working with the wood factory and got them to say they’d honor the promise they’d made earlier to deliver at least the third 1000 sets of enclosures. (This is the part where we can’t talk about some business details we really want to talk about.)

What happened next will amaze you.

Ok, fine, it probably won’t amaze you, because this wouldn’t be a Keyboardio backer update without multiple, cascading catastrophes.

The next day, our factory told us that the wood shop’s finishing and painting operation had had a fire and would be out of commission for weeks. To the best of our knowledge, nobody was hurt.

We don’t think they made this up.

Finally, this past Thursday, the wood factory told us that the finishing and painting workshop was going to be back in operation by this coming Monday and that we could sleep easy and should expect our 1000 sets of pristine-quality enclosures by “the middle of November.”

(As an aside, it sounds like there’s an opportunity for us to buy a gorgeous looking historic bridge between two of the boroughs of New York City. If we did a Kickstarter, would anyone want to get in on it?)

Plans B, C, and D

About two weeks ago, when we first found out that the wood factory hadn’t been making the enclosures they’d promised to, we started a search for backup suppliers, reaching out to more than 20 other suppliers mostly in the Pearl River Delta area (near our factory) that looked like they might be a good fit.

We got quotes from 10 wood factories ourselves. Our factory found a few more suppliers and the folks who have been helping us with project management found another couple.

At this point, we have at least five potential suppliers who will finish samples of the enclosure for us this coming week. At the same time, the factory is working through site visits at each supplier, to make sure they appear to be on the up and up.

Most of the suppliers are within a few dollars of the original supplier in one direction or the other. Most of them have quoted a lead time of around 20 days for the first 1000 sets.

Our current plan is to place orders for 1000 sets of enclosures with two or three of these new suppliers. We’ll ask each one to deliver their first 500 sets as soon as possible.

If the original factory somehow manages to deliver 1000 high-quality enclosures, we’ll ship 'em. If the new suppliers deliver before the original factory, we’ll ship those.

Once we have 1000 sets of good quality enclosures, the factory can turn a 1000 unit mass production run in 7-10 days. After that, third party QC will take a few days, then air shipment to the US for distribution should take somewhere between 5 and 8 days, depending on a bunch of factors.

Once it got to our fulfillment partner, this first mass production run took about two weeks to ship out. That’s a good deal longer than we’d expected or planned for. We’re working on a few ways to try to cut this down and get things shipped out much faster than happened for MP1.

As soon as the factory has another set of 1000 good quality enclosures, they’ll do another mass production run again. We believe that third set of 1000 keyboards will cover all existing pre-orders, possibly leaving us with a few units in stock to sell for immediate fulfillment.

After that third batch of 1000 keyboards gets shipped out by the factory, the factory will keep working on the remainder of our order: MP4, the fourth mass production run. (These are keyboards that nobody’s bought yet.) We’ve come to terms with the fact that we won’t have new keyboards in stock to sell on Black Friday this year.

We’re well aware that we’re poster children for Murphy’s Law and that making the following assertion has a decent chance of invalidating it: As of today, we still believe that everybody who has pre-ordered a keyboard will get it before December 25, 2017.

Extra keycap sets

As we’ve mentioned before, all keyboards are shipping with QWERTY keycaps. Any extra keycap sets we owe you will ship later under separate cover. The factory says that the tooling for the packaging for the extra keycap sets has been completed, but we haven’t seen samples or photos of it yet.

The factory really doesn’t want to produce the 'extra’ keycap sets until keyboard production is properly in hand.

To help control shipping costs and shorten timelines, we’re looking at shipping your keycaps to you directly from the factory. As we know more, we’ll definitely tell you.

The Keyboardio community forums

If you haven’t already checked out our community forums, head on over to https://community.keyboard.io. There’s plenty of interesting discussion about code, mounting options, keyboard layouts, and all sorts of other things.

What to do if you’re having trouble with your Model 01

Generally the reports we’ve been getting from the field are overwhelmingly positive, though, as with any physical product, a few backers have received units that weren’t as perfect as they should have been. We’ve been working with the factory to make sure that issues found with MP1 keyboards won’t be repeated during MP2.

If your keyboard isn’t behaving as you’d expect, please drop us a line at help@keyboard.io and we’ll make it right.

Seeing us in person

We’re taking part in the big Bay Area Mechanical Keyboard Meetup that’s happening on November 11th in Palo Alto. There will be a lot of keyboards there, ours and others’ and old rare ones, plus some talks and good times. The event is free, but requires advanced sign-up. Hope to see some of you there!

<3 j+k

TL;DR: QC of the first 1000 keyboards starts tomorrow. They should be on a plane to the US by Monday; The wood factory is working on the second 1000 enclosures, but hasn’t promised a delivery date yet; Jesse’s in China for an unplanned trip to do QC; Bay Area meetup November 11th.

Hello from Oakland and Shenzhen,

(Right now, Jesse’s on the ground in Shenzhen for an unexpected factory visit. More on that later.)

When we wrote last month, we’d been pretty certain that keyboards for all Kickstarter backers would be on their way by now. That did not come to pass.

As of today, the factory has completed assembly of the first 1000 Model 01 keyboards. They’re currently going through a 72 hour burn-in test.

Why only 1000? Our wood factory was having trouble meeting their deliverables schedule, so we decided to split the Kickstarter manufacturing into two batches of 1000 units. The hope is to deliver as many high-quality Model 01s as we can, as soon as we can, even if that means they won’t all be delivered at the same time.

Overall, the latest delay has primarily been due to quality issues with the wooden enclosures. As the factory checked the enclosures that had arrived from the wood CNC shop, they found a number of defects.

These defects ranged from discolored “spots”, a few cracked pieces, mis-installed brass screw nuts, and several hundred pieces where the wood seemed to have been over-milled such that there was a big gap between the baseplate and the enclosure.

The wood factory is taking responsibility for the issues and has been working overtime to rework and replace the bad enclosures.

Obviously, we’re pretty frustrated at the slowdown in production, but in the end we recognize that we’d much rather have caught these issues now rather than shipping out bad keyboards.

Now that the wood factory has delivered 1000 good enclosures, they’re working on the next 1000. After that, they’ll deliver another batch. Once those keyboards are built, we’ll be caught up on preordered keyboards.

There are a bunch of factors that will influence the order in which we send out keyboards, but one important one is the BackerKit survey. If you have not yet filled out the survey confirming your address, you should do that. If you didn’t get the survey, please write us at founders@keyboard.io.

Inspection

Typically, the way pre-shipping quality inspection for a consumer item works is that a third-party inspector does a spot check of a small number of randomly selected units from the upcoming shipment. (This is in addition to controls the factory has implemented internally.) If no serious issues are found, the shipment goes out the door.

We were not terribly happy with some of the quality issues that made it out the door on PVT keyboards. Since then, we’ve been working with the factory to help them figure out process improvements to reduce or eliminate the kinds of issues we saw. We’ve also been beefing up our documentation of the quality standard and test procedures.

The factory does significant testing as part of the assembly process. At the same time, we decided that we’re going to do a relatively expensive “100% check” of at least the first few hundred units in the first mass production run. Based on the results of that check, we’ll decided whether to continue with the 100% check or back down to random sampling.

When we contacted our third party QC firm a week ago to schedule their inspection of the first 1000 keyboards, they told us that they might have one day for us next weekend, but are otherwise unavailable. They also told us that they won’t be available for most of October.

That was… unexpected and somewhat disappointing.

Since then, we’ve been scrambling to find a replacement quality inspection firm. If the Model 01 were a ‘regular’ keyboard, we’d be comfortable letting a QC firm use their standard quality check procedures without any special training. But the Model 01 is not a regular keyboard.

Since the quality inspectors Jesse trained to check the Model 01 aren’t around, we didn’t see any other option than to put Jesse on a plane to Shenzhen. On Wednesday Jesse will start the quality inspection, along with a freelance quality inspector recommended by a friend of ours as a sort of trial run.  At the same time, we’re continuing to talk to a few other options to see if one of them might be a better fit.

Keycap sets

While in Shenzhen, Jesse should be approving the plastic packaging for the ‘extra’ keycap sets. If the packaging looks right, the tooling to produce it will take about two weeks. There’s a decent chance that the extra keycap sets will ship with some keyboards, but we’re pretty sure that we’re going to end up shipping some Model 01s to customers and then shipping your extra keycap sets as a followup as soon as they’re available.

Box of crap

If things go well with the factory, Jesse may have a few days in Shenzhen to do another run of our “box of crap from Shenzhen” project. If you think you might want a box of crap from Shenzhen, keep an eye on http://twitter.com/keyboardio. We may try shipping everything to ourselves in California and doing fulfillment from the US time around.

Bay Area Meetup

We’re taking part in the big Bay Area Mechanical Keyboard Meetup that’s happening on November 11th in Palo Alto. There will be a lot of keyboards there, ours and others’ and old rare ones, plus some talks and good times. The event is free, but requires advanced sign-up. Hope to see some of you there!

<3 j+k

TL;DR: PVT keyboards are in customers hands; we’ve been working to ensure that issues found during the PVT run will be resolved before mass production starts next week; we’re working with BackerKit to get surveys out to everyone who preordered a keyboard from our website.

Hello from Oakland,

Since we last wrote in June, we’ve shipped the first 65 keyboards to Kickstarter backers who offered to help us test keyboards from the pilot run. At the same time, we’ve been working with the factory to get ready for Mass Production.

Mass Production starts next week

Mass Production is currently slated to start on August 25. The factory believe it will take them about a week to make the 2000 units to cover our Kickstarter preorders. After that, the keyboards will be tested in Shenzhen and be packed for their journey to our warehouse in California. Last time around, that took a little more than a week.

Once they arrive, we’ll spend a day or two doing additional spot checks to make sure everything looks right, and then we’ll ask the warehouse to start shipping them out as fast as they can.

On September 21, the factory will start “MP2"—the second mass production run. That will cover all post-Kickstarter preorders, as well as a whole bunch more. Right now, they’re estimating that MP2 will be done by October 7, though we’re looking at options for shipping them from China in blocks of 500 as they’re finished and tested.

We’ve started working with BackerKit to get preorders from our website into their system, so we can send out surveys to everyone who’s preordered. If you’ve preordered from our website, you can expect email from BackerKit as soon as the survey is ready. We’ll also tweet about it.

PVT Keyboards

Generally, the response to the PVT keyboards has been fantastic. We tried shipping the keyboards without "overpacking” them into larger cardboard boxes around the boxes. Initially, we were a little bit worried that the box might not offer enough protection, so we asked customers to tweet us some pictures of how things looked when they arrived. The packaging did its job—every keyboard arrived in good condition.

The point of a PVT run is to do a limited production run so you can fine-tune the process and figure out what sorts of problems you’re going to have. Using that information, you can update your assembly and test procedures and figure out if there are any design or materials issues that need to be addressed.

As expected, the keyboards haven’t been without their issues.

The biggest assembly/test issues we’ve run into are: soldering issues, wood surface issues, and keycap printing issues.

Soldering

We knew that there were a number of soldering problems with the PVT units, mostly due to the outsourced PCB Assembly factory they used to hit the PVT deadline. Even before we’d shipped, we’d identified some potential issues and they had done a first pass of additional quality checks. On a few keyboards, there were issues they’d missed, mostly related to bad solder joints. While PVT backers have been very understanding, this is something that can’t happen for mass production and the factory has told me that they’ve switched PCBA back to their standard supplier for mass production. At the same time, they’ve been working on additional manual QC steps to ensure soldering problems don’t slip through to shipped units for MP.

Wood surface

Some PVT users have found that the right hand enclosures for their keyboards have a slightly rougher surface than the left hand enclosure. We discovered this issue when we were checking the wooden enclosures before we shipped, but decided that it wasn’t a critical issue for PVT keyboards. We’ve told PVT customers who find it distracting that we’d be happy to swap out their enclosures once MP units are available.  The reason this happened is that the wood factory made all of the left hand enclosures for PVT on one day and all of the right hand enclosures for PVT the next day. We’ve worked with the factory to document a “quality standard” for the wooden enclosures, using problems we found during PVT as examples. As the wood factory delivers batches of new enclosures to our factory, they’re doing a second check of them and sending units that don’t meet our requirements back to the vendor for rework.

Keycap printing

We’ve written before about the quality issues we’ve had with keycap laser engraving. So far, two PVT backers have reported keys swapped with each other, one backer has reported a key that had been lasered upside-down and one backer had a second ‘J’ key where his 'L’ should be. We swapped out the affected keys. Most of the keycap engraving issues were the result of a crazy manual process the factory was using to build sets of “good” keycaps from imperfect sets they got back from the laser engraving supplier. The factory has been working hard to improve the laser-engraving process. They’ve upgraded the painting jigs to make sure that the keycap alignment stays consistent on the laser engraver. They’ve built a custom tool to help make sure that the right keys get placed in the right positions for laser engraving and they’ve spent time training the technicians at the laser engraving factory to make sure that the keys are correctly placed on the jigs. We’ve also worked with them to clean up the placement of the legends a bit and slightly decreased the font size of all the key legends with words like “shift”, “num”, “return”, “alt”, etc. The factory has also been working with the keycap injection molding factory to correct some small errors in the keycap tooling.

Tight RJ45 cables

One issue some folks have run into is that short RJ45 cable require a lot of pressure to “lock” into the jacks on either half of the keyboard the first few times you do it. Other than the tight fit, the cables work fine. This is, we believed, caused by a tolerance issue between the jacks on the keyboard and the specific style of connector on the cable. The cable supplier has been looking for alternate connectors, but as yet, everything they’ve tried fits similarly. For the first mass production run, we’ll be sticking with these cables. If the factory manages to find a better option, we might change them for future runs.

Arrows of Ambiguity

There is one design issue that we’re working with with the factory to correct. The sliding dovetail bar and rail system used to connect the two halves of the keyboard has been confusing to many users. The rails are tapered to ensure that the center bars slide on easily and click into place when slid in the correct direction. We printed arrows on the rails on the bottom of the keyboard to help show which direction to slide the keyboard into the center bars. Unfortunately, the arrows turn out to be confusing for some folks and it’s possible to force the bars far enough onto the rails to get stuck if you slide them in the wrong direction. Once you understand how the center bars are supposed to be connected, it’s pretty easy to keep doing it the right way.

We’ve updated the manual to better explain how to orient the center bars and have made a small design change to the rails that makes it impossible to slide the center bars on in the wrong direction. The factory is working hard to modify the injection molding tooling for the rails, but there’s a chance it won’t be completed before mass production for Kickstarter keyboards is done. If that’s the case, we’ll be providing Kickstarter backers with an additional reminder about the right way to connect the center bars as we ship their keyboards.

Software

Our PVT users have reported a number of issues with the firmware shipped with PVT keyboards. The big ones were that Media Keys and mouse button emulation didn’t work right on macOS. There were a few additional bugs reported around key repeat and LED modes. We believe we’ve resolved all of the above issues in the Mass Production firmware, which we had to send to the factory on Sunday. Of course, one of the things about the Model 01 is that we plan to continue to improve your keyboard’s firmware for a long time to come.

We also made a change to the keyboard’s “bootloader”, which is the program that’s responsible for letting you update the keyboard’s firmware. A number of PVT users have made changes to their keyboard’s firmware ranging from changing the key layout to writing cool new LED effects. A few of them have managed to 'brick’ their keyboards, making it impossible to get into the bootloader to reprogram their keyboards without unscrewing the enclosure and hitting the handy 'reset’ button we’ve put on the PCB. Because of this, we’ve added a new feature to the bootloader: If you hold down the 'prog’ key when you first connect your keyboard to your computer, it will enter the bootloader without trying to execute your possibly-broken firmware and you’ll have a little bit of time to flash a new program.

Along with all of this, we’ve been working to clean up the firmware source code, to fix a number of portability issues in the build tool, and to improve the code’s documentation. We’ve also started to write user-facing documentation. We’re tracking the documentation tasks here: https://github.com/keyboardio/Kaleidoscope/projects/1

There are a few 'major’ features which aren’t in the shipping keyboard firmware, because we aren’t confident they’re safe and stable enough. The two biggies are iOS 10 support and “Boot Protocol” keyboard support. We will provide both features as part of a free firmware update. (All firmware updates for the Model 01 will be free.)

• Apple appears to have made a change to how iOS 10 handles USB devices that say they want more than 100mW of power. In earlier versions, iOS would warn the user that the device wasn’t supported, but the keyboard would still work. Now, iOS refuses to let the device connect. We have a version of the firmware that works just fine (but without LEDs) on iOS, but need to do some engineering to make sure that we aren’t committing horrible atrocities against the USB stack on every other device to support iOS.
• The USB specification defines, essentially, two kinds of keyboards. Regular keyboards and “Boot Protocol” keyboards. Boot Protocol keyboards speak a simplified version of the USB keyboard protocol designed to be a little easier to implement in a computer’s BIOS or pre-boot environment. The downside of the Boot Protocol is that it doesn’t support “NKRO"—you can only press six keys at the same time. Our core libraries speak both keyboard protocols, but Kaleidoscope doesn’t yet do the right thing when the host wants to talk Boot Keyboard rather than NKRO keyboard. We ended up deciding that the changes required to make the keyboard do the right thing were too potentially destabilizing to add just before locking the firmware down for mass production. Getting this fixed is a high-priority task for Jesse, and we’re hoping to have it available as an update in the near future.

As always, our firmware lives at https://github.com/keyboardio/Kaleidoscope/ and new contributors are most welcome!

Builderscon 2017

Earlier this month, Jesse gave a talk at Builderscon about our experiences finding and working with a factory. The conference is primarily intended for the local audience and his talk was one of the few in English, though the conference organized simultaneous translation so folks actually laughed at some of his jokes.

All in all, the conference was really well organized and folks seemed pretty excited about the Model 01. One of the neatest talks Jesse saw, but didn’t really understand, was the story of one intrepid hacker’s adventure designing 3D models for a difference engine and 3D printing all the parts.

There’s already a video of Jesse’s talk up on Youtube. It’s about an hour long and is a reasonable (if slightly simplified) summary of the last two years of our backer updates.

Tokyo Meetup

The day after the conference, we held a little meetup at FabCafe in Shibuya, Tokyo. It all came together at the last minute and while we tried to email backers we knew were near Tokyo, we know it was short notice.  It was lots of fun. FabCafe is a super-neat space. Their primary space is a trendy coffeebar with laser cutters. Hidden away upstairs is a proper makerspace with bigger tools and meeting space. As we got too big for the cafe, they very graciously offered us up the private space upstairs.  About fifteen people came by to check out the Model 01 and to chat keyboards.

Boston Meetup?

We’re going to be in Boston for a family event from August 28-31 and are trying to figure out if we have the free time to host a little Keyboardio meetup. If we do, we’ll announce it over on https://twitter.com/keyboardio. (Alternatively, email kaia@keyboard.io with subject line "Boston meetup” and we’ll make sure you get an email with details.)

<3 j+k

Hello from Oakland!

TL;DR: PVT assembly is done; Mass Production should start next month.

We have quite a bit of news to report this month. Probably the biggest and most exciting news is that we’re (finally) no longer three weeks away from starting the pilot run (also known as PVT). Roughly 85 PVT units were produced last week and are finishing up 72 hours of ‘burn-in’ testing now. On Wednesday, our third-party QC agency will double-check them as they go into their boxes. On Thursday or Friday, an express shipper should pick them up and put them on a plane destined for our fulfillment partner in California.

From there, they’ll be dispatched to some early Kickstarter backers.

(We know we haven’t sent your surveys yet. We’re working on the survey design with BackerKit now. Look for email about the survey in your inbox in the next week or so.)

Long time, no backer update

It’s been about six weeks since we last wrote. The first three weeks were pretty quiet, but the last three weeks have been mind-bendingly busy. The past week was really, really busy. We live-tweeted the PVT run.

The first few Model 01s start their 72 hour burn-in tests

Algernon collected our tweets from last week into a series of moments:

• Day one: https://twitter.com/i/moments/876743212114825216
• Day two: https://twitter.com/i/moments/877030769008533504
• Day three: https://twitter.com/i/moments/877393482255564800
• Day four: https://twitter.com/i/moments/877758532044685312

Jesse just returned from two weeks in Shenzhen. The plan was for him to spend a little over a week there finalizing the keycap laser engraving, checking out last minute details and supervising PVT on Monday, June 19. Things didn’t go exactly to plan, but as you’ll read, things went!

Keycap factory

Jesse’s first day in Shenzhen, he visited our keycap factory to check in on updates to our injection molds and to talk through some issues with the trays they made for the laser engraving factory to use during the painting and engraving process.

Visiting with our keycaps’ injection molds

They promised to get things fixed up right away and to produce sample keycaps within a day or two. When the caps still hadn’t shown up by Friday morning, we got a little antsy and asked what the problem was. Our factory’s team said they weren’t quite sure, since the keycap factory claimed to have already completed the modifications.

At this point, we were prepared for any number of possible explanations or excuses. We’ve had a lot go wrong during the manufacturing process and thought that we were beyond being surprised.

The factory called up the keycap supplier, who told them that the modifications weren’t done… and that getting them done that day might be difficult.

Because the factory had caught fire.

They reassured us that nobody had been hurt (and that our tooling was fine) and seemed to be up and running by the next day.

They did manage to get 120 sets of keycaps and painting jigs done in time for PVT, though the painting jigs appeared to still be of the older design that was prone to warping. Our factory doesn’t know why the keycap factory hasn’t delivered the newer painting jigs, but has promised to get to the bottom of it.

Keycap laser engraving factory

On his second day in Shenzhen, Jesse and the team from the factory braved typhoon Merbok to pay a visit to the keycap laser engraving factory in Dongguan. The primary issue on the agenda was figuring out why the keycap labels kept ending up significantly offset from where we’d put them.

We’d been sending files to the factory that showed the key labels with a variety of annotations to help the laser operator place them correctly. We’d gone so far as to build a layout that matched the painting jig to the millimeter.

Once we got to the keycap factory, the laser company took the sample keycap sets we’d brought with us and painted them black. They put them in the drying oven and suggested we go for lunch.

An engineer at the laser-engraving supplier aligns key legend

When we got back, we watched as the laser engraving technician printed out a copy of our alignment template, deleted all of the guide lines from the file, imported the legends into the laser engraving software, and proceeded to try to eyeball the correct label placement.

Going in, we were annoyed at how far off the legends were. In retrospect, it’s astonishing how good his manual placement was.

After a bit of back and forth, we found a workflow that seemed to work better. We convinced him to keep the keycap top edge outlines in the drawing and to use those to make sure he was placing the labels correctly. That seemed to work better.

PCB Fabrication

The factory’s regular PCB fabrication company wasn’t able to make the boards in time for PVT, so they ended up using another supplier we recommended. These are folks that friends of ours have used on numerous occasions. Instead of the 20 days the factory’s regular supplier quoted, these folks quote 6-7 day lead time. Well, they usually quote 6-7 day lead time. This time around, they quoted a 10-12 day lead time. Twelve days would have had the boards done on the Thursday before PVT began. But on Thursday, they told our factory that the earliest they could promise delivery was Monday. When the factory pushed back, they said that if we didn’t like the lead time, we could try another supplier.

The factory escalated to a customer service manager who said that they could do Sunday, but that was the best we’d get. Sunday delivery meant that PCBA wouldn’t start until Monday, which would delay PVT. This was not ideal, but we seemed stuck. Asking around a bit, some friends mentioned that the supplier had been “closed by the government for some reason.”

We reached out to Ken Li, our amazing project management consultant. Ken said “What’s our order number? Hang on a minute.”

Ken came back about 10 minutes later saying that he’d reached out to the CEO of the PCB factory, whom he happens to know, and that the CEO had promised our PCBs would be delivered on Saturday. In the end, he was as good as his word and the boards arrived late on Saturday.

PCBA

The factory’s regular PCBA (PCB Assembly) shop is located in the same building as the factory. Unfortunately, they don’t work on Sundays and the factory couldn’t get them to budge, so, in an effort to catch up to the promised PVT schedule, they reached out to another supplier who do work on Sundays. This was the first time they’d worked with this supplier. And, as of last Wednesday, it is the last time they will work with this supplier.

This diode should never have made it through the PCBA shop’s QC process. Consequently, the factory won’t be using this PCBA shop again

Every single PCB had problems. Most of the issues were mis-soldered diodes. The boards looked like they’d been assembled by hand, rather than by machine. They also looked like nobody had done a simple visual inspection after assembly.

Visual inspection is a standard part of commercial PCB assembly. Any halfway decent vendor has automated equipment to do visual checks of boards after assembly.

The factory ended up having to “rework” every board to fix surface mount assembly issues. This ended up being a two day setback. In hindsight, we would have been better off if they’d used their regular SMT shop, like they’ll be doing for mass production.

Type C Cables

We thought we had a workable Type C cable for PVT, but when the factory placed the order with the supplier, they admitted that they didn’t actually have the cables or the required raw materials to make more in stock and that we’d be looking at a lead time of 3 weeks or more.

Plan B was to wake up early on Saturday morning and go to Huaqiang Bei, the world’s largest electronics market and buy some cables retail for PVT. Ken offered to come along and to bring a friend who was a professional cable maker who wanted a chance to pitch for our business.

Over coffee, he convinced us that we really, really didn’t want to buy retail cables in the market and said he could build cables to our spec, on our deadline.

He ran off to find some samples of a few options. When he got back, we talked through our requirements and he promised to send samples.

Four days later, we had three sample cables that looked like we expected. Shortly after Jesse headed home from Shenzhen, the vendor delivered all the cables for PVT.

PVT production

Jesse didn’t tell the factory it was his birthday, but they still delivered quite the birthday present. The factory fired up the assembly line and produced all of the PVT units on Wednesday, June 21.

We’ll talk more about the assembly line in a future update.

QC

The next day, Jesse sat with the factory’s QC team and our third-party QC agency to review the assembled units and to start to identify what quality issues they need to look out for and what we generally consider acceptable. For the sorts of parts they’re used to dealing with, this is pretty easy. For some things, like the wooden enclosures, we needed to be a bit more detailed.

This wooden enclosure failed QC because the edge had been sanded too much, rendering it flat, rather than rounded.

Wood is, of course, a natural material without a uniform color. We showed the factory examples of unacceptable problems, which included tooling marks on visible surfaces, wood filler on visible surfaces, and ugly discolorations. At the same time, they pointed out issues they thought we’d reject. For the most part, we agreed with them.

Model 01 keyboards that have passed our QC checks

It was a pretty long day, but we managed to go through about 70 keyboards. The way it worked was that we’d choose a category of defect to review (like “keycaps” or “wood”) and then queue up each keyboard. The factory’s QC team would check the keyboard and either pass it or reject it. If they rejected it, they’d add a little arrow sticker pointing at the problem. If they passed it, they’d hand it to our QC guy. He’d review it, either adding a sticker and rejecting it or pass it and hand it to Jesse. He’d review the keyboard. If Jesse didn’t pass it, we’d stop the line, discuss the defect and the QC team would modify their acceptance criteria.

At the end of a given pass through the keyboards, we’d all review the rejected keyboards together to make sure they weren’t being overly critical.

This “birdseye” Model 01 is the one Jesse gave Kaia for our 10 year wedding anniversary. (If you look closely, you can see a keycap that failed QC and had to be swapped out.)

The one surprise for everybody was “birdseye figure” Maple. Our jaws dropped when we found a few examples of birdseye maple in with the PVT keyboard enclosures. They were gorgeous. The factory’s QC team thought they looked funny and didn’t look like they were supposed to. It took a bit, but they now understand that if any birdseye Maple makes it into the supply chain, they shouldn’t reject it. (Right now, we’re not able to promise anyone a birdseye figure enclosure. It’s very much luck of the draw.)

At the end of the night, the factory packed us up a carton of six keyboards, which Jesse hand-carried back to California, where we’ll be doing a little bit of additional QC.

In-house QC

Issues we found during PVT

By far, the biggest issues we ran into during PVT were with the quality of laser-engraving on the keycaps. At some point, we’ll do a writeup of the root cause of the keycap issues, but that’ll be a few thousand words on its own.

Checking every keycap by hand

We ended up with an eye-watering defect rate of around 25% on the laser engraving for the keycap sets. The two biggest failures were poorly aligned legends and keycaps that had been laser-engraved backwards. The proximate cause of these problems was that the keycaps need to be moved (manually!) from one painting jig to another in order to make sure the bottom edges of the keycaps get good paint coverage. Because this process requires a human to touch every single keycap, it’s a point where errors get introduced.

These are a subset of the keycaps that failed QC. The factory lined them up and counted them to help drive-home the magnitude of the problem to their suppliers

Needless to say, this isn’t going to work for mass production.

We’ve spent a bunch of time with the factory going over process improvements for mass production. They want to do a bit of research and testing before deciding on a final solution, but we’ve got a few promising options.

What’s next?

Now that PVT production is “done”, there’s still a lot to do before you get your keyboards. Thankfully, almost all of it is straightforward for the factory.

Once we get the PVT units to the states, we’ll ship them out to willing early backers. (There will be a question on the backer survey about it.) After that, we’ll be eagerly awaiting feedback from folks who get PVT keyboards to make sure that we’re not missing something obvious.

The factory will be subjecting a few Model 01s to torture tests. They’ll be exposing them to a few days of ultra-bright UV, spraying them down with a saline fog, and dropping them on a concrete floor. While we sort of wish we could be there to watch these tests, we’re probably better off not having to watch the torture taking place. That should take about two weeks.

The factory will also be sending one keyboard to the FCC test lab to send onward to their partner lab in Taiwan for final testing and issuance of the “Declaration of Conformity”. That should also take about two weeks.

At the same time, the factory is working to refine their internal test and assembly process based on the PVT run. That’ll let them build Model 01s faster and more reliably when they have to make a few thousand of them at a go.

Right now, we’re expecting the first couple thousand units to roll off the line in July.

<3 j+k

TL;DR: Initial FCC/CE pretesting results; Component sourcing craziness; We’re guessing that we’re about two weeks out from starting PVT

Hello from Oakland,

When we last wrote, we were pretty sure that April would see Jesse in Shenzhen overseeing manufacture of the first 100 Model 01 keyboards.

That didn’t happen and we’re pretty bummed out about it.

As of this writing, we believe Jesse will be getting on a plane to Shenzhen in two or three weeks. Below, we’re going to lay out what’s happened in the past month and what still needs to happen before we start manufacturing.

RJ45 connectors

When we wrote last month, we told you that the factory had enough RJ45 connectors for the first 1800 keyboards in their warehouse. While that is now true, it wasn’t then. We only found this out about two and a half weeks ago, when the factory told us they hadn’t yet sent keyboards to the test lab for FCC/CE pretesting.

As we dug into it, we found out that they hadn’t sent the keyboards to the test lab because they couldn’t finish building the circuit boards. They couldn’t finish building the circuit boards because, while he factory’s purchasing team thought they had received a large order of our RJ45 connectors, in fact the Chinese customs authorities had received them.

It took a while for the whole story to come out, but it sounds like what happened was that the customs declaration for the Taiwan-made RJ45 connectors wildly undervalued the product, so much so that Chinese customs took one look at the packages, said “Nope!”, and impounded the goods.

We don’t know why the customs declaration was so wrong, though we suspect that someone, somewhere was trying to be “helpful” to minimize taxes paid on the parts. We got somewhat cranky when we found all this out, and while we don’t know exactly what went down, the factory reports that they have 3600 RJ45s sitting in their warehouse now.

FCC/CE pretesting

As we mentioned above, the first round of FCC/CE pretesting didn’t happen when the factory said it would. Once we understood what was going on, the factory scrambled to buy a small number of RJ45 connectors to build test keyboards to submit for lab testing.

For reasons that still aren’t entirely clear to us, the factory didn’t send sample keyboards to the test lab until April 24.

On the 26th, the test lab told us that they were short some documentation: electronic schematics, an electronic block diagram, a copy of the Bill of Materials, a datasheet for the LEDs and an “Operational Description.” We turned all of that around in a couple of hours, though we had to guess on the operational description and nobody had ever asked us for a block diagram for the keyboard before. The test lab promised to have results back in two business days, though this week’s May Day holiday meant we didn’t see the first round results until Tuesday.

We almost passed.

Results from our FCC test report. The red circles show electro-magnetic emissions we need to reduce.

Of course, in the realm of certification, “almost passed” means “failed.” The Model 01 had just a little bit too much electromagnetic radiation in the 35-115Mhz range.

Because this sort of thing is pretty common, and a little bit of a dark art unless you have a lot of experience, we designed the Model 01’s circuit board to let us add slew rate limiting capacitors on the USB data lines.

One of the services the test lab offers is engineering support to help resolve exactly this sort of issue. The next step is for them to evaluate our proposed solution and make their recommendations for a remediation. It sounds like we’ll have their feedback by the end of this week.

If the next round of pretesting looks good, we can start building the electronics for the PVT units in a little over a week. We’ll turn around and use some of those PVT keyboards for the official CE certification and FCC Declaration of Conformity.

Keyswitches

The factory currently has enough keyswitches in stock for the first 500 keyboards. (They’ve counted them twice.) Due to a mistake by their purchasing department, the rest of the switches didn’t get ordered until February. Matias, our keyswitch manufacturer, initially thought they’d be delivering all of our switches sometime in March, but that didn’t end up happening. They’ve committed to delivering us at least 1000 keyboards worth of keyswitches per month from here on in.

The delay is related to an improvement the they made to reduce the amount of “wobble” in the switches at the end of last year. The new switches feel a lot nicer, but due to the tighter tolerances, their manufacturing operation has taken some time to come up to speed. The team at Matias is bending over backwards to get us switches as quickly as possible and right now we believe that we’ll have covered 90% of pre-orders with June’s keyswitch delivery.

Keycap injection molding

When we last wrote, we told you that the keycap factory was reworking all of our keycap stems, so they didn’t grip the keyswitches so hard that they ripped the keyboard apart when you try to pull one out. That work was supposed to be completed by, roughly, the first week of April. It ended up stretching another two weeks. We got test keycaps in the mail at the two weeks ago and didn’t get the factory’s test report until the a few days later. By our measurements, 48 of the keycaps required somewhere between 1.3-1.5 kg of force to remove, with the rest at 0.9 kg to 1.2 kg. 0.9 kg is way, way too loose. 1.2 kg would be workable, but isn’t what we’d expected.

The factory’s measurements all come in a bit lower than ours, with 20 of the keycaps at less than 1 kg of force being required to remove them and no keycap requiring more than 1.2 kg of force to remove. We think the difference can be chalked up to us using a fancy digital force gauge and them using something a bit more analog.

After a bit of negotiation, the factory agreed to have the keycap factory fix the keycaps that they measured as taking less than 1 kg of force to remove. They’d promised us an estimate of the time required to to modify the tooling two weeks ago, but we’re still waiting on it. Thankfully this kind of modification is relatively quick and easy. To get the keycaps to fit more tightly, the plastic key stems need to be a hair bigger. And to do that, the factory needs to shave a little bit of metal off the injection molds. (To reduce the force required to remove the keycaps, they had to cut big holes in the injection molds, weld in new steel and remachine the molds. That was the bit that took so long last time.)

Keycap laser engraving

In mid-April, the keycap engraving supplier sent updated samples back to our factory. They’d done a pretty good job of getting the legends and paint in the right place, but even from the photos, we could see that there were a few labels that were still misaligned. The factory promised to have the laser engraving supplier take another pass at it. That new version of the keycaps was due to be returned to the factory two weeks ago, but didn’t show up until yesterday.

We’re starting to suspect that the only way these issues are going to get ironed out is to have Jesse go sit at the laser engraving supplier’s office and work with their engineers. If that happens, you can expect a bunch of photos in the next backer update.

Center bars

In the last update, we told you that the factory said they’d resolved the issues with the center bars deforming a bit as they came out of the injection molds. They sent us samples. They had, in fact, resolved the curvature issue. From there, we pointed out a few issues we saw with the finish on the samples and asked them to spend a bit of time cleaning up the surface of the injection molds. Two weeks ago, we got new samples in the mail. We’re happy enough with the cleanup work they’ve done to declare the center bars “done”

Short RJ45 cables

One of the small, but important details we’ve been working through has been the RJ45 cables we use to connect the two halves of the keyboard.

The longer cables were pretty easy to sort out, but the shorter cables have been unexpectedly difficult. Most cable vendors didn’t seem to get why we needed our short cables to be so short. (They have to fit inside the little cutouts between the two halves of the keyboard.)

We specced 12 or 13cm long cables, end to end. We got samples back that were half that length and a couple that were double that length. Some of the samples had connectors that were so long they couldn’t fit inside the keyboard. One or two were so stiff that they put serious lateral pressure on the RJ45 jacks. That could lead to the jacks becoming less reliable over time. One sample had connectors so short that they got stuck inside the jacks. One sample had shrouds on the cable that were so long that the connectors wouldn’t even fit into the jacks.

One vendor has made us a new set of sample cables that look like they’re going to be just right. They match some samples that were made by a different vendor last year. Last year’s cables fit great and worked well, but weren’t particularly well made. We expect to have the new vendor’s version in hand sometime next week.

USB Type C Cables

Last time around, we mentioned that we were looking at possibly forgoing the “right angle” Type C cables if we couldn’t find a vendor who made something that seemed high enough quality. In the end, all the right-angle Type C cables we found that were well made had gigantic plastic housings around the connectors that prevented them from getting a solid connection to the Model 01. All of the good-looking right-angle cables were either mechanically or electrically wrong.

We’ve ended up deciding to go with a straight Type C cable from a vendor our factory has worked with before. It’s still 1.5 meters long and our teardown shows it to be well made. It has a 56K pull-up resistor in the right place, which means it’s not fraudulently claiming to support USB Power Delivery, so it shouldn’t melt when you use it to charge your phone.

In the end, it’s a pretty boring cable. We’re ok with that.

Software

On the software front, there’s lots of great stuff happening.

Algernon has been hard at work on Chrysalis, the cross-platform desktop interface for managing, programming and flashing the Model 01 (and other keyboards that run Kaleidoscope or have similar serial communications protocols.)

Over the past month, he’s built functionality to autodetect a supported keyboard when you plug it in, flash new firmware on the keyboard over the USB line using @noopkat’s avrgirl, the beginnings of a graphical LED theme editor, as well as a few more bits. He’s released a first “alpha” version which includes a mock keyboard, so you can try it out, even without a supported keyboard.

You can follow along with his progress on GitHub: https://github.com/algernon/Chrysalis, and can read a recent progress report on his blog.

Since Algernon has been focusing on the GUI, the firmware itself hasn’t seen a ton of work over the past month. (It doesn’t hurt that we have implementations of pretty much every keyboard feature we’ve reached for.)

The other exciting development on the firmware front isn’t exactly Keyboardio related. About a week ago, Wez Furlong popped up with patches to Kaleidoscope that let him add support for keyboards with ARM microcontrollers and Bluetooth connections. One of his first targets is Adafruit’s nRF52 Bluetooth board.

(Click here to see a demo)

There’s still a fair amount of refactoring to go before it’s clean and easy to use Kaleidoscope with ARM chips and Bluetooth connections, but we’re really excited to see this happening. If more keyboards can run Kaleidoscope, more intrepid hackers will be building cool plugins to make your keyboard better.

Scheduling

Our current best guess is that Jesse will be in Shenzhen in about two weeks to oversee the PVT run.

Sometime right around then, Kickstarter backers should expect to get email asking them to confirm their shipping addresses and keycap preferences. While we already have shipping addresses for post-Kickstarter preorder customers, we’re planning on sending them surveys, too. We know it’s been awhile since folks ordered and we want to make sure your Model 01 doesn’t end up shipped to the wrong place.

PVT assembly should take one week. After that, most of the PVT units will be express-shipped to customers. We don’t have a firm methodology for who gets the PVT units. Our plan is still to send those units to early Kickstarter backers located in the United States who confirm on their surveys that they’re happy to get some of the earliest production keyboards, and to put them through their paces, but we’re going to play it a little bit by ear.

A few of the PVT units will be held back for reliability testing. The factory thinks that will take about a week and a half. If reliability testing of the PVT keyboards looks good, the factory will start producing your keyboards as fast as they can.

<3 j+k

TL;DR: Plastic parts are looking good; Wooden parts are looking good; Raw material costs in China are skyrocketing; Open source is amazing.

Hello from Oakland!

This is part two of our March 2017 update. If you missed part one last week, you can read it on Kickstarter.

Baseplates

One of the ongoing problems we’ve had with our manufacturer is the quality of the plastic baseplates being produced for us.

First, here’s a recap of the saga of the baseplates up to now.

When we started this process, we’d specified metal baseplates. We’d also specified press-fit nuts and standoffs for anchoring the keyplates and wooden enclosures to the baseplate.

The factory proposed swapping out the metal for plastic. While we were a little bit nervous about this, plastic has the dual advantages of weighing less and better fitting our design. With the plastic baseplates, the factory was able to mold in features to help protect the RJ45 and Type C jacks. They were able to mold the metal tripod mounts right into the plastic.

And back when we thought there were going to be flip-out feet on the bottom of the keyboard, they were going to be able to hide the feet inside baseplates.

Well, it turned out that the flip-out feet just weren’t going to work. The factory took a gamble and had their partner start on the baseplate injection molding tooling before we signed off on this part of the design.

The first versions of the baseplates came back with numerous “hot points” where you could see sinking due to the plastic cooling at a different rate. That’s not great, but is entirely normal for a first (called a “T0”) injection sample.

Entirely aside from the issue with the feet, the quality of the baseplates provided by the factory’s partner was not up to snuff. There were many issues, both small and large. The biggest issues centered around the quality of the tooling. The internal “ribbing” on the parts looked like they’d been hand-cut by a CNC operator. The lines were too thin and not quite in the right place. There were places that two of the ribs were supposed to touch, but instead they were about a millimeter apart. Because the structural stability of the part depends on those ribs being fused together, this was less than ideal.

We asked them to try again.

The injection factory blew their self-imposed deadline by a week or two. The parts they came back with looked a little better, but as we looked at the details, we found that the structural lines hadn’t really been made thicker. They’d just cut new lines next to them which… almost lined up.

This is right about when the factory read them the riot act. They told the injection supplier they had one more chance to fix things.

The injection factory blew their self-imposed deadline by a week or two. The next version was a little better, but still just didn’t match the design files sent to them in a number of places. What’s worse was that the injection factory thought they’d done a decent job.

By now, it was early January and we were at T5 or T6. We were pretty frustrated.

The factory relented and asked the injection molding factory to take one more shot at things.

The injection factory blew their self-imposed deadline by a week or two. The samples they came out with were ok, but still not production quality.

This was the point at which our factory started shopping for a new injection molding partner in earnest. Just before Chinese New Year, the factory told us which of the three bids for the baseplates they thought we should accept. The only real details we had were the pricing and the factory’s opinion, so we ran with it.

They told us that the new factory expected to finish tooling the week of March 5, with samples sometime that week.

Once Jesse got to China, the factory took Jesse to visit the new baseplate supplier. They confirmed that tooling and “T0” injection samples would be ready on March 8.

The tooling supplier showed us the work-in-progress injection molds, as well as their fancy test equipment.

Well, March 8 rolled around and the injection factory asked if we could delay our visit until the afternoon. With a little bit of trepidation, we agreed. When we showed up sometime after lunch, we were brought straight to one of their injection machines. The factory manager, in a suit, was pulling out baseplates, “hot off the presses.”

The first “T0” sample of the left baseplate from the new supplier

We took them upstairs to the injection shop’s offices, pulled out our sharpies and started marking up issues as we found them. There were a reasonable number of “hot point” sinkholes, but that’s very much part for the course as molds get dialed in.

The first samples didn’t have tripod mounts on the bottom. To add them, it was just a matter of adding tripod mount inserts into the tool as it made each sample. We then found that the tripod screw inserts, while exactly as long as we’d specified, were missing a bit of headroom above them. The problem with this is that longer tripod screws wouldn’t be able to screw all the way into the baseplates, leaving them extra wobbly. We’d picked a depth of 6mm because that’s what the ISO standard for camera tripod mounts says. Perhaps unsurprisingly, some of our test commercially available camera tripods actually have screws as long as 7 or 8mm. After talking things through with the injection supplier, we decided to switch to 10mm long threaded inserts. There was a little bit of stress because the injection factory’s supplier claimed they couldn’t actually do 10mm inserts while still making them from stainless steel. We’d have to switch to brass or galvanized iron. This seemed a little bit nutty to us. We agreed that brass was the better of the two choices. (While it might deform a little bit, it’s unlikely to start rusting after repeated use wears it down.) The next day, with help from our friends at HWTrek, we got the injection factory to recant and switch back to stainless steel. We’re still not quite sure what happened, but we’re happy with the result.

Back to the baseplate inspection. These samples looked good. Surprisingly good. The baseplate supplier agreed to run the samples through their fancy 3D measurement machine to compare them to the CAD designs. We handed them our marked-up samples showing off the issues we’d found.

A report showing how well the first version of the new baseplates matched the CAD designs

At that point, it was again time to talk about texture. The injection factory pulled out their texture sample card. We settled on texture number 48, which is a moderately dense texture we found both attractive and a little bit “grippy.”

After that, the factory’s R&D manager marked up samples with a white paint pen to show the surfaces that should have the texture applied. The injection molding factory said they expected to be able to turn new samples fixing the substantive issues by the time Jesse left China four days later. They warned that we shouldn’t expect the texture to be applied yet.

Lo and behold, four hours before Jesse left China, a team from the factory met him at his hotel with a fully assembled sample keyboard. Including new textured samples of the baseplates. We’re very happy with the new samples, though the factory found a few small things that needed touching up. The baseplate supplier is making those changes now.

The stands

We didn’t know this until we turned up at the factory, but the supplier that the factory selected for the baseplates is also the supplier making the stands.

The story for the stands is much the same as the story for the baseplates, starting from the injection mold factory’s promise date of March 8. Shortly after the sample baseplates came popping out of the injection molding machine, sets of stands started to accumulate on a work table.

We went through the same process as with the baseplates, looking for hot points and other molding issues.

If you look carefully, you can see some small issues with the stand circled in green marker

One thing that came up was that the screw bosses on the top halves of the stands were inconsistent sizes. We figured this out as we tried to assemble a stand. One or two screws fit well, but most of them just spun in place, never finding any plastic to bite into.

Another problem was that in the top half of the stands, a bit of the support structure seemed to be a mite short. This resulted in the part feeling slightly less sturdy.

The molding factory promised to run the stands through their measuring system, too.

We marked up a sample stand, showing which parts should be textured and which parts should be left flat so the adhesive for the feet and top piece would get a better grip.

Four days later, the factory presented us with a pair of stands with just about every issue corrected.

One item we’ve found since is that the screw hole in the stands is…not quite a circle. Our best guess is that the mechanical engineer modeled the hole as a cylinder, but started from the wrong “ground plane.” In testing, this issue doesn’t affect the keyboard’s usability or stability at all (nor is it visible when the keyboard is on the stands), but it makes us a little bit sad to have missed when it was still an easy fix.

Screws for the stand

The tripod screws we’re using for the stand have been working pretty well, but the factory thinks they can be improved.

Apparently, the way all-metal screws like this are made is that they’re CNCed. Because of this, the cost difference between a “standard” screw and a custom screw is minimal. They tell us that the lead-time for this new screw is similar to the lead time for the “standard” design, too.

The factory had a sample of a custom screw made and is sending it along to us for our review and (possibly) our blessing.

Center bars and rails

The center bars and rails are the bits that keep the two halves of the Model 01 connected in a flat or tented configuration. You don’t have to use them, but many of us want our keyboards to be one-piece units most of the time.

The design of the center bars and rails has been fixed for months, but the factory has been working through fit and warping issues.

These parts are going to get a lot of weird stresses put on them. On the factory’s advice, we ended up deciding to make them out of glass-filled nylon rather than ABS+PC. Glass-filled nylon should be more resilient.

As it turns out, the factory has had a hard time getting the nylon versions of the bars to lie flat. When they’ve come out of the molds, they’ve ended up a little bit warped. This is… less than ideal for a part whose role in the world is to be flat and keep another part flat and aligned. The factory’s been able to make nice, flat versions out of ABS and ABS+Polycarbonate for months. With the glass-filled nylon version, they’ve been stuck tweaking the injection molding machine’s pressure, cycle time, and other parameters.

They even went so far as to build a straightening jig to fix the parts as they come out of the mold. It works similar to a how a set of braces helps a poor tween’s teeth end up nice and straight. This seemed like a bit of a cop-out to us, so we asked around and found out that this is a relatively common strategy for parts like this that have warping problems.

These are the straightening / flattening jigs

In the past couple days, the factory finally sent over pictures of glass-filled nylon center bars that look to be perfectly flat. They say that the solution ended up being a change to the percentage of glass in the nylon.

Previous versions of the rails that screw onto the bottoms of the enclosures have been a little bit loose, resulting in unnecessary wobbling. They’ve gotten a tiny revision to make sure that they fit snugly into the center bars.

We’re excited to test new samples of both parts when the factory sends them to us in the next week.

Keycap laser engraving

One of the most frustrating aspects of the preparation for mass production has been an ongoing series of quality issues with the painting and laser-engraving of the keycaps.

The first paint and laser supplier the factory was working with promised turnaround times of two to three days.

The first keycap samples they sent back had the key labels centered on the keycaps, when we’d specified that they should be anchored to the lower left hand corner.

The second samples that came back had the per-key alignment pretty much correct, but had put the left hand key labels on the right hand and vice-versa. Some labels ended up misspelled, which must have taken serious work, since we’d sent “camera-ready” copy.

The third samples just had… sloppy alignment. It looked like they’d gone in and modified our key layout to change the spacing between the symbols on a few keys and a number of keys had their labels engraved at a funny angle. A few keys had been engraved…upside down. We noticed that the sides of some of the keycaps didn’t have great paint coverage.

This was the point at which we decided to stop trying to get the labels right until Jesse was on the ground in Shenzhen.

Early in the trip, he visited the keycap laser engraving factory in Dongguan, the next city over from Shenzhen. Their operation seemed pretty professional and they appeared to know what they were doing. Our questions about how things had gone so badly wrong got kind of weird answers, but the supplier asked that we give them a chance. They promised to carefully check every sample before they sent it back.

While there, we took the time to ask them about the UV clearcoating they spray onto the keys to protect them after laser engraving. We asked if we could pay extra for higher-quality UV clearcoat “paint”. The factory boss said that we weren’t going to see any real quality increase from changing it out. We asked about whether they could do multiple coats to protect your keycaps better. The boss said that they already do two coats. When we asked “how about three?”, he told us that we’d be wasting our money and that two was really totally sufficient. We left with the understanding that they’d paint and engrave new samples and get them back to us in a few days.

It wasn’t until a day later that we found out that this was a new laser engraving supplier and that the factory was ditching the first supplier.

Well, the promised keycaps didn’t show up after two days. Or three. Or four. Or five. And then they finally showed up with exactly the same problems we’d seen before Jesse met with them.

After a very tense meeting, we finally figured out that somebody in the factory’s logistics department had sent the keycaps out to the old supplier again.

Early this week, the factory finally sent us photographs of newly-engraved keycaps from the new supplier. They look much, much better. We see a few small issues, though we can’t properly check them until the factory sends them to us.

One ongoing issue has been the quality of the painting toward the bottoms of the keycaps. This is because the “painting jig” for the keys places them too close together. When we asked why the painting jig packs the keys in so tight, we were told that it was designed to match the injection mold for the keycaps. And indeed it does. As best as we can tell, the keycap factory economized on steel when designing our mold, which has led to cascading problems down the line.

There’s a second issue with the painting jigs. Because they’re built without tall side-walls and ribbing, they have a tendency to, you guessed it, warp. This means that it’s much harder for key labels to end up aligned correctly. This issue, at least, is easy to fix. The jig maker is currently modifying the jig design.

Our factory tells us that the painting and laser engraving supplier has committed to taking responsibility for high-quality painting and engraving of all keycaps. As we understand it, the current process has the painting and laser engraving supplier’s staff manually moving keycaps between jigs to make sure there’s plenty of space for the paint to coat the keycaps. On top of this, the factory has committed that their QC team will reject any keyset with an error.

This is, somewhat understandably, stressing us out a bit.

RJ45 cables

The two halves of the Model 01 are connected by an 8P8C RJ45 cable, known to most of the world as an “ethernet cable.”

We’ve gone back and forth a number of times with the factory in our quest to get decent cables. The first few suppliers they reached out to had a hard time actually coughing up any sample cables at all, which isn’t exactly a great sign for a cable company.

One of the first samples we got back had such long cables that it couldn’t actually fit inside the Model 01.

Eventually, the factory found a cable supplier that seemed both willing and able to make us the cables we wanted.

We’d asked for samples of three different weights of cable: 24AWG, 26AWG and 28AWG. The goal was to find a cable that was heavy enough to get the power from one half of the keyboard to the other, but not so heavy (and stiff) that it was unwieldy. We ended up settling on the 26AWG cable weight for the 1M long cable.

The problem we’re dealing with now is the short (10-15cm) cable. When we use 26 AWG cabling, it’s stiff enough to put lateral stress on the RJ45 jacks on your keyboard when the two halves are connected by the center bar. If we ship with this cable, it will shorten the life of your keyboard. We’ve asked the factory to work with the cable supplier to find a more reasonable option. We showed them a “flat” cable as one possible solution, but the cable vendor nixed it, saying that they’ve seen far too many quality issues with those flat cables. We’re hoping to hear back about the factory’s meeting with the cable vendor in the next day or two. This isn’t a major issue and should be pretty easy to resolve, but we have to actually, you know, resolve it.

USB Type C cables

We’ve previously talked about the Type C cable we thought we were going to use. Its construction was good. It met the part of the USB spec so many cables cheat on. (It had a 56k resistor to identify itself.)

There’s just one problem with it that we didn’t catch. Its Type C connector is big and blocky and… slightly out of spec physically, meaning that it doesn’t mate cleanly with the Model 01 when the circuit board is inside the enclosure.

The factory is talking to the supplier, as well as other possible vendors. The long and short of it is that we may end up deciding to ship with “straight” USB A to Type C cables, rather than cables with a right-angle Type C connector. At the end of the day, a good cable with a solid connection is more important than the cable shape.

In a previous update, we mentioned we’d rejected a good-looking Type C cable because its internal resistor violated the USB specifications. One of us accidentally started to use the sample to charge a phone. The cable did a good job of reminding us that it was out of spec. It got so hot that the glue inside melted and leaked out!

Long lead-time electrical components

We’ve previously mentioned that the APA 102C LEDs we’re using are in short supply. We have enough on hand for the first 900 or so keyboards, but are still waiting on delivery of the rest of the order. The factory has a call in to the manufacturer to get a firm delivery date, but as of now, we have no reason to suspect that the LEDs are going to cause a delay in mass production.

The factory has about 500 keyboards worth of keyswitches on hand, with the balance to start showing up over the next month. This is later than we’d expected, but shouldn’t have a measurable impact on production. The factory had told us that their purchasing department was on top of switch procurement. When we asked them to confirm, they said that they had all of our switches sitting in their warehouse. When we asked them to confirm again, they realized that they’d made a mistake and only had 10% of the switches they thought they did.

The RJ45 jacks we’re using are another item with a bit of a longer lead time. These, at least, got ordered before Chinese New Year. The factory has enough of them on hand for the first 1800 keyboards and say that they’re expecting the balance of the order any day now.

As of now, the factory believes that none of the other electrical components they don’t have on hand have a lead time longer than a couple weeks.

Wooden parts

While in Shenzhen, we paid a visit to our wood supplier to check in on timelines for PVT, as well as a few special projects.

When we asked about the lead times for the enclosures for the first 100 units, we got a pleasant surprise. “Oh. Those are done. We have them in the warehouse. Do you want to see?”

This was tied to another surprise. They’d gone ahead and made 100 sets of the stands out of wood. We hadn’t asked them do and hadn’t signed off on them, but they did it as a favor to the factory to help cut down timelines. They weren’t actually thrilled about making the parts, since the “organic” contoured shapes are very slow to CNC.

There are a couple issues we’re working through with the stands they made. The big one is that in some cases, their post-CNC sanding was a little bit aggressive, so some of the stands have a few edges that are a little flatter. We’ve asked them to go through the sets they’ve made and cull out the worst of the lot.

Our original plan had been to ship wooden stands with the PVT units because the plastic stands weren’t going to be ready. Given the other schedule slippage, that’s no longer an issue.

On the one hand, these are parts we didn’t order. On the other, we really like the wood factory and they’ve done a lot of spec work for us over the past two years.

The current plan is to offer them as a special-order after-market accessory. More on this front as it develops.

After that, we talked a bit about the lead times for mass production. It sounds like the first 2000 keyboards worth of enclosures will take 3 weeks from when we say “go.” The current plan is to say “go” once PVT units start arriving in folks hands and they verify that everything looks right.

Costs

One thing we haven’t talked a whole lot about over the course of this project has been money. There are a bunch of reasons for that. Many of them center on us feeling uncomfortable talking about money in public.

We’ve been pretty frugal throughout this whole process. While we’ve had outside help, we haven’t hired full time staff to work with us. Instead, we’ve brought in contractors for shorter gigs as we’ve needed their skills. We pay ourselves, but at nowhere near market rates. There have been times when we could have used more of your money to speed the project up, but we judged that doing so was more likely to hurt us down the road.

The upshot of this is that we’ve been in a very healthy cash position leading up to mass production.

One of the reasons we’ve been so conservative about cash is that it’s been our plan to use most of the Kickstarter and preorder profit to invest in inventory via a larger initial order with our factory.

That’s a very, very normal plan for a crowdfunding campaign. Heck, it’s a big part of the “kickstart” in Kickstarter.

We’ve been finalizing everything with the factory, now that the keyboard’s design is final, tooling is nearing completion, packaging has been designed, and accessories have been selected.

Even before Jesse went to Shenzhen, we’d been hearing stories from reputable sources about costs and lead-times for raw materials in China going up.

Even when you know it’s coming, a big increase in costs is never a welcome turn of events. The updated pricing we got from the factory just before Jesse left China was a good deal higher than the update we’d gotten before Chinese New year, spiking more than 20% due to materials costs. Some of the biggest jumps came in the costs of bare PCBs and the packaging itself. In the case of the packaging, it’s not just that materials costs have gone up. We’ve ended up with something nicer and more protective than originally planned.

But really, everything from aluminum to ABS pellets to cardboard has gotten a bunch more expensive. The only thing that (thankfully) hasn’t spiked is the price of the Canadian maple the enclosures are being milled from.

The factory is working to negotiate some of these increases with their suppliers and says they’ll have an updated proposal for us this weekend.

On the one hand, this was a pretty big shock and we’re pretty unhappy about it. On the other, the reason for most of the increases is something we support. China’s been in the process of tightening environmental regulations and significantly increasing enforcement of those regulations.

Cardboard boxes are a lot more expensive than they had been because, we’re told, the Chinese government has shut down a number of paper mills that were polluting the environment more than they should have.

So, where does this leave us?

We’re still figuring that out.

We’re not screwed.

This does not impact our ability to ship your keyboards.

It may or may not impact the number of keyboards we order for sale (rather than presale) to customers, as well as our ability to sell the Model 01 through channels (like brick and mortar boutiques or other websites) because there’s less margin to go around.

While we’re a bit stressed out about this, you don’t need to be. We’re being very careful to make sure that we still have enough cash on hand after ordering your keyboards to pay for shipping and to pay our taxes.

Right now, we need to sit tight and wait for the factory to finish negotiating with their suppliers. We know that the negotiated costs will go down a bit from the last quote. Once that’s done, we have folks who have been doing this for many years helping us out with vetting and negotiating the quotation. They’ve already been doing spot checks of a couple items and are digging into a few issues for us.

Firmware

There’s this particular feeling for an open source project author. It’s somewhere between pride, awe and abject terror.

We’re new parents, so we’re not entirely sure, but we suspect that it’s the same feeling you get when your kid moves out of the house.

In November 2013, Jesse created a trivial keyboard firmware for our prototype keyboards. He wrote it in an evening. While watching a movie. After having had a drink or two. (Jesse insists that while he was intoxicated when writing the first version of our firmware, he “never does this”)

Over the course of our time at Highway 1 and through the progression of prototypes we’ve built, he tweaked the firmware, always planning to throw it away before keyboards ever got into customer hands.

Last October, this guy showed up on #keyboardio, our IRC channel, to chat about the Model 01’s firmware. He quickly started to submit small patches to fix obvious bugs. And then we started talking about “advanced” features.

Before we knew it, Algernon (Gergely Nagy) had built some libraries for the Model 01’s Firmware. (At that point, we might have been calling it “Arduino-Keyboard” or “Model01-Firmware”). But he quickly found that the stuff he wanted to do wasn’t easy to do in the firmware structure we’d built.

He created his own firmware for the Model 01 called Akela. It had all sorts of neat features we really wanted in our firmware.

It should be noted that at this point, he didn’t have a Model 01 prototype, but it was clear to us that if anyone in the world should have one, it was him. It took a little bit of work, but we got FedEx to deliver a keyboard to him in Budapest on December 23. You can read about it on his blog: https://asylum.madhouse-project.org/blog/2016/12/24/the-package/

That’s when things really started to take off.

We somehow suckered Algernon into porting all of his cool code back into the Keyboardio firmware, developing a plugin infrastructure, and then splitting out just about all the functionality into plugins.

Before we knew it, our firmware had a couple of dozen plugins spread across the “Keyboardio” and “Akela” namespaces.

Algernon mentioned that there was another keyboard maker he’d been talking to who thought they might be interested in running our firmware on their board. He asked if we’d have a problem with that.

We don’t have a problem with that. It’s pretty much the validation of everything we’d hoped for with the firmware. The more keyboards running our firmware, the more users we have. That directly drives more people to be interested in hacking on the firmware, resulting in better firmware for everybody.

This did make one thing abundantly clear. Calling the firmware “Keyboardio Firmware” was no longer right. We wanted a name that references Keyboardio and/or the Model 01, but was broader and more inclusive.

We ended up settling on Kaleidoscope. A kaleidoscope make an infinite number of beautiful and unique works of art from a bunch of shiny baubles. Plugins are like shiny baubles, especially when they control neat LED effects.

As it happens, “kaleidoscope” is also one of the collective nouns for butterflies.

After we settled on the name, Algernon went and renamed all of our code and all of his code into the shared namespace.

To date, we have more than 40 firmware plugins for Kaleidoscope. You can read an overview of many of the plugins in Algernon’s blog post: https://asylum.madhouse-project.org/blog/2017/03/10/rose-tinted-glasses/

Over the past couple weeks, he’s been churning out new functionality so fast we can barely keep up.

First was AlphaSquare, which uses the Model 01’s LEDs to draw ASCII characters on your keyboard in a 4x4 pixel font.

Then came Focus, our long-promised bidirectional serial communication protocol. Plugins can register themselves with Focus to either send data back to your computer or let themselves be configured or updated remotely.

Over the past couple weeks, Algernon has been working on moving keyboard layouts and LED effects out of static program memory into EEPROM. What that means is that you’re now able to use Focus to change onboard key layouts without needing to change the Model 01’s program. Based on an offhand comment from Jesse on IRC, Algernon extended this support to let you remap a keyboard’s layout from the keyboard, using a scheme similar to the way older Kinesis Advantage keyboards did it.

One of the cool things he’s put together that he hasn’t announced anywhere yet (but said we could talk about here) is the first version of a tool that changes the Model 01’s keymaps based on your current application. His first implementation is for Linux, but the concept is very similar to a prototype we built for macOS a while back, so we’re pretty confident that ports for more platforms should be pretty straightforward.

Next up, it sounds like he might be working on a “fingerpainting” mode for LED lighting effects or letting you “record” macros on the keyboard.

Chrysalis

Of course, not everybody wants to hack their keyboard from their keyboard or to use a serial console to change their layout.

We’ve long promised a GUI tool for configuring your keyboard. As recently as a month ago, we were 100% sure that this was going to be something we built…a bit later, after we’d had some time to recuperate from shipping your keyboards.

Well, Algernon and @thebaronhimself (Simon-Claudius Wystrach) got to chatting about things on IRC and somehow Simon-Claudius ended up volunteering to start building the desktop GUI.

So far, our only contributions have been suggesting the name and some encouragement.

It’s very much early days, but Simon-Claudius sent over this animation of the core of the SVG+Javascript code that will let you use your mouse to configure your keyboard. We’re very, very excited.

A work-in-progress view of Chrysalis

What’s next

Right now, we’re working on getting the manufacturing and shipping logistics set up while we wait for the latest sample parts from the factory and the factory waits for the keycap tooling supplier to finish fixing the issues we talked about in the last update. As we wrote last time, the ETA for all of that to be wrapped up is early April.

If all goes according to plan, the next backer update should find Jesse back on the ground in Shenzhen doing quality control checks on the first 100 Model 01s to roll off the line. If that’s not the case, we’ll be back next month with more exciting stories from the trenches.

<3 j+k