Make sure they're fully staffed (probably redundant but it needs to be said). Also, check around to see if there are any traffic jams on the island and see if you can create any extra roads to clear them up; this will help your builders get to their destination faster (not to mention any teamsters on their way to the dock). Finally, don't build too far away from a road; builders will need to walk out to them THEN walk all the way back to town, typically. Failing that, I will tell you bluntly that the builders -- particularly early in the game -- take their damn time getting around to projects and getting them up. Hell, at times it's all I can do not to execute a few of them in the hopes that it'll get the rest to get their ass in gear. As they gain skill they will get faster but again there will be times when just none of them are working. I strongly suggest getting a second construction office as soon as you have a labor surplus; that will cut down on a lot of the downtime. Also, don't be afraid to use the quick build option if you need something up now and you have the extra money (though always leave enough to pay your bills until the next ship/aid relief). Finally, I do recommend grabbing the Quick-Dry Cement DLC: the cement factory gives a boost to construction speed. It's a small boost but it's also an obscenely cheap factory that doesn't require blueprints, resources, or electricity.
Sychotic Quote for the Whenever: "Don't think I'll go easy on you just because you're a dog."