Bazel and Golang

As a Google software engineer, I use the internal Blaze build system every day. As far as build systems go, it’s really good. So I figured I’d give Bazel a try, since it’s basically the same thing.

Basically.

The part I guess I forgot about was that it isn’t supported and developed by teams of paid engineers all the time. That said, it’s actually not bad. It just feels incomplete after coming from Blaze.

I’m partway through setting it up to build an existing project in Go, which is just barely supported. The documentation is minimal and unclear, but I think I can help out.

Here’s how to take a $GOPATH-type workspace and make it work with Bazel without mucking up all of your project paths, and without symlinking parts of one workspace into another. Because that’s actually what the documentation tells you to do, and it feels wrong to follow it.

First, install Bazel. I’ll wait.

All done? Oh, you don’t like this writing style? Neither do I. I’ll stop now...

Next:

1. Go to your $GOPATH (the parent of bin/ pkg/ and src/) and clone the Bazel repository:
   git clone github.com/bazelbuild/bazel
2. Now we need to make some changes under src/:
   cd src/
   ln -s ../bazel/tools .
3. Write a WORKSPACE file:
   cat >WORKSPACE <
   load("@bazel_tools//tools/build_rules/go:def.bzl", "go_repositories")
   go_repositories()
   EOF
4. Write a BUILD file:
   cat >BUILD <
   load("@bazel_tools//tools/build_rules/go:def.bzl", "go_prefix")
   go_prefix("")
   EOF

I’m not sure why the tools symlink is necessary; it may be a bug in the Go rules. They can’t seem to find their automatically-added dependency without it.

And that’s it, besides for writing BUILD files for each of your packages. I do have one more tip for that part, though: Write your BUILD files in the parent directories of your packages. If you do this:

src/github.com/me/mypackage/BUILD:

load("@bazel_tools//tools/build_rules/go:def.bzl", "go_library")

go_library(

name = “mypackage”,

srcs = [“mypackage.go”],

visibility = [“//visibility:public”],

)

Then when you use that package from Go, you’ll need to double the ‘mypackage’ part:

src/github.com/me/pkguser/BUILD:

load("@bazel_tools//tools/build_rules/go:def.bzl", “go_binary”)

go_binary(

name = “pkguser”,

srcs = [“pkguser/pkguser.go”],

deps = [“//github.com/me/mypackage”],

)

src/github.com/me/pkguser/pkguser.go:

package pkguser

import (

“github.com/me/mypackage/mypackage” // Redundant, eh?

)

But if you write the BUILD file one level up, the names work out as you would expect:

src/github.com/me/BUILD:

load("@bazel_tools//tools/build_rules/go:def.bzl", "go_library", “go_binary”)

go_library(

name = “mypackage”,

srcs = [“mypackage/mypackage.go”],

)

go_binary(

name = “pkguser”,

srcs = [“pkguser/pkguser.go”],

deps = [“:mypackage”],

)

src/github.com/me/pkguser/pkguser.go:

package pkguser

import (

“github.com/me/mypackage”

)

On the other hand, your BUILD files will be longer mix together multiple unrelated packages if you do it this way, so you’ll need to decide what is best for your own project.

Choosing a Computer Monitor

Computer monitors are difficult to choose because you need to pay attention to so many different features, and a few mis-features.

I played no part in my wife’s recent choice of a new monitor, and we both regret it. I stayed out because she didn’t ask for advice (I tend to give too much), and she didn’t ask because she didn’t know about the Rules of Monitor Selection.

Yes. There are Rules.

They are not, as I had assumed, common knowledge.

Allow me to let you in on the secret. Firstly, do not buy any monitor...

• With an external power supply. They are hard to replace when they wear out, and they add clutter.
• With a power input other than a standard PC power cord. Anything else is hard to replace.
• With display or power inputs perpendicular to the screen; they should enter straight up from below. When they stick out of the back, they add depth that won’t be accounted for in the published dimensions, and they add a twisting stress to the circuits inside from gravity pulling on the cords.
• Missing any input type you need right now, for obvious reasons.
• Without an HDMI input (DisplayPort if you prefer Macs). Your next computer will probably have this kind of output.
• Without a height adjustment. Your desk may not be the perfect height, but at least your monitor can be.
• With a resolution below 1920x1080 (aka full HD or 1080p) in either dimension. This is the current minimum standard. Remember when it was 800x600? It sucked, right? Same thing here. Don’t buy below the beast.

Completely avoid anything that matches those. Don't even look at them. They will try to seduce you with pretty pictures, but they're not worth the heartache the next morning.

Now to help you choose the best from what remains. A monitor is better if, in generally descending order, it...

• Has a DisplayPort input. This is the next most popular after HDMI. If your next computer doesn’t have HDMI, it will probably have DisplayPort.
• Has a pitch adjustment (tilt up/down). You can use the height and pitch adjustments to get rid of a reflection and keep the screen in a comfortable position.
• Has a swivel base (twist left/right), so when you turn the screen to let someone else see, you don't knock over your coffee.
• Includes a built in USB hub (bonus points for USB 3).
• Can rotate between portrait and landscape modes. This is most useful for coding or writing long documents.
• Is color calibrated, for graphical design.
• Does not have built-in speakers. They will be awful. This is a monitor, not a TV.

What about 4k monitors? Make sure your computer can support the higher resolution if you’re thinking about getting one of these. It needs either a DisplayPort output, or HDMI 2.0 or newer.

So what do I recommend? I have an ASUS PA249Q 24.1” 1920x1200 monitor. It has HDMI, DisplayPort, DVI, and D-SUB (aka VGA) inputs, no speakers, a standard PC power cord, all the various adjustments, a 4-port USB 3.0 hub, and multiple different color calibrations. It is easily the best monitor I have ever had, and I strongly recommend it. It can even do side-by-side and picture-in-picture.

Oh, and you should probably avoid Samsung. They tend to feel really cheap.

Note: As an Amazon associate, I may earn a commission from purchases made through links followed from this page.

Prototypes

If you’ve ever visited a Google office and used the bathroom (great way to begin, right?), you probably saw an episode of Testing on the Toilet (or possibly Learning on the Loo) posted at eye-level. This week I saw a post about prototypes. It said there is a simple rule to determine whether your project is a prototype: Whether it will live a short about of time, and then be deleted.

But that’s not an easy way to identify a prototype. “Why not?” you ask (well, probably not, but it fits my writing style). “It sounds simple!”

No, it’s not simple, because it requires you to see into the future. You need to know that your project will only live a couple weeks, and then go away. But how could you know that?

While working on a personal project (I am implementing the Raft consensus algorithm in Go), I discovered a much simpler test for whether a project is a prototype: If you cannot use test driven development, it is a prototype. Think about it for a moment. I’ll leave a break between the paragraphs.

Really, think about it...

When can you write the tests before the actual code? When you understand the solution, which means you have a plan.

When you’re ready to write an implementation.

When can’t you write the tests first? When you are exploring the problem, trying to understand what the solution might look like.

And guess what? If you try to write tests after writing an exploratory solution, the tests are going to suck. You’ve probably done it before. They cover the easy, obvious cases, but you still worry a little about some corner cases that were just too hard to reach. Maybe you were considerate enough to put todo comments on them, but a todo is not the same as a test.

My experience implementing Raft convinced me that I am writing a prototype. I will do it again when I finish, and I may do it again after that. This is how I learn; I write something that kinda works, and then I know enough to write it better. Eventually better is enough to be production code.

Stack allocation for Java

In my previous post, I mentioned how useful stack allocation would be for Java, and expressed my surprise, bordering on exasperation, for why it is not available.

Since then, I've thought about how such a thing might work, and realized it is impossible to include without either breaking Java's security model, or throwing away any benefits through additional checks.

First, let me describe a possible implementation: We define a new stacknew operator which acts just like the new operator, except that it returns a reference to an object allocated in the current stack frame, instead of on the heap.

Now for a few design choices, what I would choose, and why:

1. During construction, should all instances of new act like stacknew? No, They should act like normal new. Although it might be convenient for such an automatic conversion to stacknew during construction, it would likely cause problems:
   1. It would encourage the use of stacknew to construct objects not intended for stack-frame lifetime, through behaviors like saving references to themselves in other objects.
   2. It would bring up the question of converting all new instances to stacknew when executed by the object, which would be impossible to get right. Should we convert method code? Static method code? Inherited code from superclasses? Taint all calls made by the object to other objects? It is too messy and not predictable.
   3. Therefore, stacknew must be intended for use on classes specifically designed for it.
2. Should it be possible to determine if the this object was allocated with stacknew? Yes. This will allow special-purpose code that is stacknew-aware to modify its behavior, using calls to stacknew instead of new when it is stack-allocated. Otherwise it would be difficult to write a general-purpose class without passing around a boolean flag indicating which new operator to use. In fact, Object should have a new method, isStackAllocated(), probably public.
3. What should happen when dereferencing a reference to a stack-allocated object whose stack frame returned? This should generate a NullPointerException, or similar. Maybe a new Error subclass would be called for. And this is where the trouble starts...

To fit with Java's model of simply disallowing any sort of unsafe memory access, it must not be possible to successfully dereference a reference to a stack-allocated object whose frame has returned. Well, that doesn't sound so hard! I hear people thinking. But it is hard. Surprisingly so.

So how do you know, when you try to dereference a pointer, whether it points to an allocation in an invalid stack frame? You cannot depend on any property of the memory in that frame. Here are the possible scenarios when trying to perform an access:

1. The object is still there and fine. This is what you would expect during a dereference operation.
2. The frame returned, but no new frame has overwritten that part of the stack yet. This could be considered lucky, but it would be a nasty bug to fix if something changed.
3. The frame returned, and a smaller function overwrote part of the memory but not all of it. This would likely produce some very indeterminate behavior; what if you grabbed a reference out of the corrupted object and then tried to dereference that? Some way be valid while others are not.
4. The frame returned, and other functions completely overwrote the object.

So how do we protect against this? As I said, we cannot trust any of the object's memory. Here is the solution I came up with:

1. Each stack frame containing any stack-allocated object will have a non-zero cryptographically secure identifier (random number) at a known offset from the beginning of the stack frame.
2. Every reference will have a type, either heapnew or stacknew. Probably a bit flag somewhere.
3. Every stacknew reference will contain, in addition to the object pointer, a pointer to the beginning of the stack frame and the value of the stack frame's identifier.
4. On every dereference, the JVM will check if it is a stacknew reference. If so, it will first verify that the frame still exists before finishing the dereference operation. If not, it will throw an exception or error. This verification is performed by matching the frame identifier in the reference to the value expected by looking at the location where the identifier would be if the frame was still alive.
5. Any time a stack frame returns, the identifier field in the frame is set to zero.

Why cryptographically secure? Because otherwise it would be theoretically possible for an attacker to guess what the identifier of a frame might be, and arrange to have a specific memory layout occur, resulting in successfully dereferencing a malicious pointer. This scheme makes it vanishingly unlikely to have a false positive match, malicious or otherwise.

I am reasonably confident that this solution will prevent accidental dereferencing of invalid memory, but it does not sound at all efficient, especially when you account for references needing to be three times their current size.

So, to conclude this surprisingly long-winded post, I am saddened to say that I seriously doubt Java could support a stacknew operator.

But that doesn't stop me from wanting one...

C++ vs. Java

I'm not trying to start any sort of religious war here; I'm writing this because I've been programming in C++ for the past six or seven months after more than three years of professional Java (and more than ten years in personal projects). So I feel like I have an informed opinion.

Unfortunately, my opinion may incense some: Programming in C++ after being used to Java is like slogging through mud. And here's why:

• Auto-completion. Eclipse does it badly for C++, and neither Emacs nor Vim will do it automatically. I'm sure I could set it up, but it still would not be as good at it as Java in Eclipse. I am unaware of other choices (don't even try to suggest Visual Studio; nobody uses Windows for real software development).
• Header files. I need to write one of these, with ancient C include guards around it, for almost every interesting piece of code.
• Prototypes. I need to repeat myself. I need to repeat myself. It's 2014!
• Surprisingly, flexibility of where classes can be written. In Java, if I want to write a new class, it pretty much always gets its own new file, and the name of the file is the name of the class. But in C++? It can go anywhere, including in whatever file I'm writing when I decide I want it. And there is no naming requirement for linking headers with source files.
• Threading is a special case in C++.
• Memory management. I wish there were more options in Java (stack allocation, for example), but it requires too much extra effort in C++, including destructors. Rust has some interesting ideas around this, but it adds to the number of indirect reference types.
• Boilerplate code. Java has some, certainly, but C++ has so much more. Refer back to header files and prototypes. And memory management (std::unique_ptr<actualtype> anyone?). And how many times have you written "virtual ~ClassName() {}"? Or, even worse, "virtual ~ClassName();" in a header and "ClassName::~ClassName() {}" (with its multiple instances of repetition) in a source file?
• Namespaces. Not only do I need to include something, but I still need to either type out the namespace every time, or add a using directive. And then we get back to problems with header files; can't put a using directive there without screwing everything up when someone includes the header. Oh, and they're not implicit based on the file path, so at a minimum I'll need to write each new one twice. Is it obvious yet how much I hate to repeat myself?

But, to be fair, there are a few areas where Java is severely lacking:

• Memory  management. It is really hard to write a no-gc Java program, and it may be impossible without abusing class fields and sacrificing immutable types. Stack allocation would go a long way here. Seriously. Why is this still missing? You could probably do it with a single new keyword, or by overloading the meaning of one of the operators.
• Const. I really miss this in Java. Really, really </Shrek voice>.
• Templates. C++ mostly got this right with weak typing. Java requires strong typing, and you can't really specialize. And they're erased, so you're doubly... Triply screwed.
• Macros and conditional compilation. Less important (to me) than anything else here, but sometimes I really wish I could define a macro that would evaluate into some repetitive multi-line construct that, instead, I need to type out (usually with the help of block-selection and editor macros). Oh, look at that, more repetition...

It's entirely possible that most of this boils down to how expressive the language is. Either way, maybe it's time to learn Go...

Insta-gin

The insta-gin bitters are coming out much better than I expected. I tested them today, and was beyond pleasantly surprised.

First I used a chopstick like I described in my last post, to smell and get a little taste. It was so much more gin-ey than expected that I just had to try a bit more.

I used a small spoon to put a few drops into the bottom of a shot glass, and then topped with vodka. It really tasted like gin! A little bit weak (easily remedied next time), but surprisingly good.

Then I made it into a gin-and-tonic, which was a mistake. It was a little weak already, and it could not stand up to the other flavors in the glass.

Next up (probably tomorrow): A gin martini.

Which reminds me that we're out of olives...

Tasting bitters

Getting a feel for how bitters taste is important, especially when making your own (but also I am realizing I should have done more with the ones I've purchased).

The difficulty, though, is that bitters are, well, bitter. And they are strong! Strong enough that you can't really taste background flavors.

So here are three ways to get an idea of the flavor of bitters:

1. Put a drop onto your palm (use a chopstick if you are macerating your own in a wide-mouthed jar). Rub your palms together, and then cup them and smell.
2. Put a little into some unflavored soda water and taste that.
3. Just taste a little straight. This might give you an idea of the strength, but you probably won't get much out of it, though, and it will probably wreck your ability to taste anything for a few minutes.

As my bitters macerate (steep), I am tasting them with methods one and three every day. I use a chopstick to get a drop to smell, and then I lick the chopstick. So far, the insta-gin has too much orange, so I removed the bit of peel, and the gentian root is really really bitter; the longer it sits, the stronger it gets and the longer it lingers.