Mac OSX Web Developer Challenges

The point of this blog post is not to bash Mac OSX.  I like my new Macbook, my wife loves her Macbook even though it's getting quite old, I just bought one for my son, and OSX has many advantages over Windows.  However, since switching to OSX away from Windows for web development I have experienced some frustrating problems.

Most IT organizations doing web development use Windows for developers and Linux and/or Windows for their servers, although as I pointed out in a previous post Apple hardware is becoming more common.  Of course, Apple got out of the server business around 2010 and there's no sign of them jumping back in.  There are actually some hosting companies that will rent out OSX in the cloud, but that's not very common, and it's not an area that is making much money for Apple.  So, those of us doing web development on Apple computers are kind of an exclusive group.

The vast majority of Mac users are not going to run into the kind of problems I am having, because they are not web developers using development tools and languages not created or supported by Apple.  I do a lot of programming in Java and Javascript.  This is a programming blog, so I am assuming the audience is other programmers, but if you are an OSX and/or iOS developer doing ObjectiveC or Swift, maybe you are in the enviable position of having everything basically always work as you expect.  That's not the world I live in, at least not in my current job.

One of the first difficulties (I won't even call it a problem) that I ran into with my new Macbook is that there are so many ways to install software on OSX.  

1. App Store (note that somebody also created a cli for the mac app store)
2. DMG (disc image) files
3. application files
4. archives
5. Windows-like installers
6. Macports for installing ports of Linux and Unix software
7. Fink, an alternative to Macports that provides a Mac implementation of Debian Linux tools like dpkg and apt-get, concentrating on pre-compiled binary packages to improve performance over Macports
8. Homebrew, another alternative to Macports.  Homebrew does not like the other competing package managers, it takes over /usr/local and doesn't like other installers to put anything there, and, similar to Macports, installs can be excruciatingly slow when it has to compile from source.  However, even with the drawbacks Homebrew seems to be better than Macports or Fink and makes it much easier for a developer to install certain software.
9. SDKMAN, for installing Java and Groovy SDK's

To be fair, there are multiple ways to install software on Windows and the various flavors of Linux as well.  However, I am just pointing out that using a Mac is generally expected to be simple and intuitive, but that is probably not going to be the case if you are a web developer.  Even if you don't want to run any software ported from Linux, there are at least 5 ways to install software on a Mac.

Some problems I have experienced as a web developer are particularly frustrating because they seem to be very specific to particular versions of OSX.  Therefore, other developers I work with may not see these issues even if they are also using a Mac.

Recently I have been working on a Node.js app that uses something called deasync.  Now, I don't want to digress into a conversation about Node.js, but suffice to say that deasync is a C hack that changes the way Node.js behaves at a low level and should probably not be in our project, but it is, and I have to deal with it.  Unfortunately, it turns out that deasync gets errors on my Macbook.

What is most frustrating about this Node.js problem that I am having is that it doesn't happen under Windows or Linux, and it doesn't seem to happen on all versions of OSX either, so basically I am the only person on my team having this problem.  And the only workaround I can find for this problem is to completely avoid it by running the Node.js app from inside an Ubuntu Linux VM.  The permanent fix should be to stop using deasync, but that would require a lot of re-writing on a project that is running out of budget.

Now, this deasync/Node.js bug is not Apple's fault, it's not even the fault of the Node.js developers.  The author of deasync doesn't want to take responsibility for it, because the error actually occurs in libuv, which is used by Node.js and deasync.  However, the libuv developers would probably say it is a problem with OSX.  I could try to fix libuv myself, it is open source, but I am a bit outside of my comfort zone with such low level C code.

Of course, these are just a couple of examples of the kind of difficulties one may run into.  I would really like to know how many web developers use Macs in their work compared to Windows and if they tend to run into similar issues or if I've just had a bit of bad luck this time around.  Also, I am hoping that with increased familiarity I will just know how to avoid problems in the future.

Can One Write Too Many Comments?

A friend of mine recently was watching a youtube video where somebody said coders should not comment their code.  My friend blamed this way of thinking on the TDD "fad".  His solution:  javadocs.

I asked my friend if he had read anything by Robert C. Martin and suggested the book "Clean Code".  He said the name was not familiar to him.  He said he doesn't read a lot of books but gets most of his ideas from the internet.  One exception he noted was a book called 21st Century C, which I am not familiar with.

Anyway, this argument about comments has been going on for as long as people have been promoting TDD (Test Driven Development) philosophy, and perhaps even longer.  When Martin Fowler wrote his book on Refactoring in 1999, one of the first books that mentioned JUnit, he categorized comments as a "code smell".  To be fair to Fowler, he said comments are a "good smell", but that they are often used as "deodorant" to to cover up for bad code.  So, too much comments may be a sign of problems.  This idea was just one part of the "extreme programming" mindset, which represented a paradigm shift which was difficult to fully accept but brought many advantages for developers.

Lean principles and my own experience tell me that too much effort placed on activities that do not directly impact the end product, such as comments (and documentation in general) can be a waste of time.  (It suddenly struck me as a bit ironic that I'm making a statement like this while writing a blog post which itself may not be the most productive activity, but whether anybody else reads this or not at least I can justify that this is an exercise useful for helping me organize my own thoughts and think through my own opinions.  Perhaps the time spent on documentation is kind of like that.)

Years ago I had a professor that insisted every single line of code should have a comment.  I have never met anybody else that wouldn't agree with me that this is a wasteful extreme.  I also have worked on projects where every function and/or file had to have a header comment, every variable had to have a comment describing what it was, and/or every change to a file had to be explained in a comment.  I even worked on one project where the rule was to never delete code, you always just commented it out.  People running these project obviously felt their rules were important, but I did not agree.

I have also worked on waterfall projects where business analysts took so long to determine requirements that there was not enough time left for actual coding the project.  I would rather produce something useable within budget constraints than try to create something perfect that never gets finished.  As they say, "perfect is the enemy of good."

In Clean Code, Martin acknowledges that Javadocs are good for describing a public API, but goes on to say that most other comments are bad and devotes several pages to support this assertion.  Has Martin gone too far?

Back to my discussion with my friend, we agreed that extremes are generally not good.  In fact, I think everybody, including managers who sign off on software projects, intuitively know that, and so I think the naming of "extreme programming" is sort of unfortunate.  XP practitioners labeled their own ideas as out of the norm, perhaps even unproven or risky.  I think they were going for the "coolness" factor calling to mind "extreme sports".  Perhaps they would not have gotten anybody's attention without the cool name, but in the long run "agile" and "scrum" sound more positive and safe.

I think comments can be helpful when they relate why a developer implemented code in a certain way.  I often put in TODO or FIXME comments in code when I know there is probably a better way to do something, but I just do not have time to explore it further.  Sometimes when I copy code from examples I find on the internet I like to put a comment giving credit to the original author and/or a link to the web site where the code came from for future reference.

However, I do not believe in hard and fast rules for such things.  I understand why managers create such rules.  I have been in management roles myself, so I know how much pressure there is to succeed and how hard it is to ensure some level of quality while getting things done quickly and minimizing risk.  But from a programmer's perspective what we run up against is that each new project has a new set of arbitrary rules.  One project might implement Uncle Bob's rule of almost never writing comments, while another project goes to the other extreme.

As programmers we just want to do a good job, but we don't want a stifling set of arbitrary rules.  We value creativity and individuality.  The agile manifesto states this quite clearly.  So, some things should be a topic for discussion amongst programmers and not something that should be imposed from the top of an organization or even by tyranny of the majority of programmers in a team.  I would argue this is one of those cases.

My First Experience With An All Javascript Project: Angular, NodeJS, and Mongo

I recently wrapped up a 9 month web application project using NodeJS and MongoDB for a backend and Angular Material on the front end.  I had a little Angular experience before, but many of the other technologies on the project were new to me.  It is quite an interesting experience taking on a lot of new technologies at once.  It can be kinda frustrating at first, but overall it was a fun project and I'm glad to have worked on it.  Anyway, I thought it would be a good time to share some of my thoughts about the technologies we used.

MongoDB

One of the most controversial decisions for this project was the use of MongoDB.  The company the project was created for has a long history with Oracle databases.  Some people have a vested interested in continuing that tradition.  However, I was tasked right at the beginning of the project to justify the use of Mongo.

It is quite easy to find articles on the internet about all the reasons that you should never ever use Mongo for basically anything period.  These articles can be kind of scary for some people, and if you have personal reasons to not want to use Mongo it is quite easy to point to these types of articles and say "see, I told you so."

Unfortunately, you can easily find negative articles like this about any technology you can name.  Whatever it is, somebody out there hates it enough to write an article about how it is inherently bad and deserves to be shunned by everyone.  This has been the case ever since the famous "Goto Statement Considered Harmful" article published before most of us were born.

Suffice to say that in the end Mongo was not in the least harmful to our project.  On the contrary, it was a key in allowing us to get a lot of functionality out the door in what was record time for the company in question.  Almost nobody thought we could do what we did, and I credit a big part of it to the flexibility of MongoDB.

MongoDB was a great fit for a NodeJS project, but also it allowed us to evolve our database in an agile fashion rather than designing it all upfront with ER diagrams and without a single argument over column naming standards.  It was a breath of fresh air compared to many other projects I have worked on.

NodeJS

Some of my feelings about NodeJS are summed up pretty nicely by Gavin Vickery, although I'm not sure about his conclusion that Python is a better solution.  I don't have a lot of Python experience.  Although I keep hearing about how Python is so popular, currently the 4th most popular language behind Java, C, and C++, according to the TIOBE index, while Javascript sits at 8, if I search for jobs in my area for each of these languages, there are a lot more javascript jobs that Python.  And there are significantly more NodeJS jobs than Flask jobs.  So, while some developers are confortable sticking with their little niche technologies, I am concerned a lot more about job security and marketability.

Anyway, the biggest problem I had with NodeJS is "callback hell".  This is the term often used for the difficulty in thinking in terms of asychronous code in Javascript.  I found that people have so much trouble thinking in these terms and getting the code right that they tend to look for ways to get around the problem rather than learn to do it the right way.

An example of somebody trying to get around doing callbacks the right way is a project on github called deasync.  The original author of deasync says on github that it is "...just a hack and I'd strongly recommend not to use it for anything."  The deasync project relies on C++ code that can cause strange errors depending upon the OS and version of NodeJS.

Another example is a project called node-fibers.  I thought at first that node-fibers seems like a more elegant approach than deasync, but it solves a slightly different problem, and it also relies on C++ code to do its magic.

I think it is better to stick with the standard Javascript way of doing things than to try to hack a solution with C++ code which could easily become invalid with some new version of NodeJS.  However, that really means you should bite the bullet and deal with "callback hell" until some future version of Javascript solves the problem.

Alternatively, if you do not like asynchronous Javascript, write your backend using another language such as Java or Python.

Restify

We decided to use Restify on this project rather than Express.  I don't have much to say about it other than it is pretty easy to use and worked fine for our project.

AngularJS

When we started our project, Angular 2 was still in beta, so we stuck with Angular 1.  The biggest problem we had with Angular was performance problems around digest cycles.  This has to do with two way binding from browser DOM to and from the model.  For sample/toy applications that you see in typical tutorials you won't run into any problems, but as an application grows the computation of keeping the DOM and model in sync can become a significant bottleneck.

Angular 2 tries to improve performance, but on future projects we are considering React instead of Angular.

Angular Material

The decision to use Google's material design had it's pros and cons.  Of course, in the future if we go with React we can no longer use Angular Material, but we can still use material design with some other library.  Angular Material was fairly limiting.  It didn't always behave as one might expect.  More often as not this is because the Material Design spec itself is so limiting.  So, we ended up supplementing with jquery controls.  For example, we did not use the Angular Material calendar but the one from jquery-ui wrapped in our own Angular directive.

The Material Design look and feel was originally crafted for Android devices, and it feels a bit strange elsewhere.  It doesn't look anything like the native look and feel you would expect on any given OS, and a lot of business users will not be content with the limited selection of controls it provides.  I don't think I would use it on my own personal projects.

Macs Still Don't Suck: IT Professionals Increasingly Prefer Apple

In the last couple years I have noticed an increase in IT departments being willing to purchase Macbooks instead of Windows laptops.  It turns out that there is evidence beyond anecdotal experience that, according to a 2015 article, Apple Macs are replacing PCs across enterprise at ‘unprecedented rate,'

It wasn't that long ago that people were writing articles informing us of all the reasons Why Mac Sucks, which impassioned others to counter that, no, actually Macs Don't Suck.  Not to be out-done by Apple, Microsoft invented Windows 8, and now the number of "Why Macs Suck" articles on the internet is totally eclipsed by "Why Windows 8 Sucks" articles.  Windows users and IT departments have so little trust in Microsoft that they regularly refuse to upgrade to newer versions of Microsoft's software, instead preferring to stay 2 or 3 major versions behind.

I have mostly used Windows, but recently got a Macbook Pro.  This prompted some of my colleagues to ask why I would want such a strange machine.  After all, it has a weird keyboard with the control key in the "wrong" place, and there is supposedly no right mouse button (or so they thought until I showed them otherwise).

Long-time Windows users are sometimes surprised and annoyed that Apple keyboards and mice and other things are slightly different from what they are used to.  However, Microsoft has for decades copied ideas from Apple, so it would be a bit silly to expect Apple to then turn around and blatantly copy Microsoft.  If Apple had copied what Microsoft did with Windows 8 it would have been a huge mistake.

It should also be noted that before Microsoft made Ctrl-C mean "copy" under MS-DOS it already meant "abort", and it still does mean that if you are typing at a command prompt within Windows.  Microsoft could have used Alt-C (the precedent set by the once popular WordStar word processor).  Then there would also not be this inconsistency between Windows and MacOS.  (On a side note, the Emacs editor did not introduce "Cua Mode" (to allow C-c, C-v, and C-x) until version 22.1.1 around 2007.)

I not only have experience with MacOS X and Windows, but also with Unix and Linux.  Depending upon who you talk to, MacOS X is either the best of both worlds (between Windows and Linux) or the worst of both.  There is certainly room for differing opinions, but I think the truth is somewhere in the middle.

Some people still think of MacOS X as Apple taking something that should be free and charging money for it (i.e. Linux).  However, MacOS X is not Linux, does not have Linux under the covers, and actually has nothing to do with Linux.  Both just happen to be related in some way to Unix, an OS that few people actually use anymore.

In fact, Linux is not even a complete OS.  Linux is merely a Unix-like OS kernel.  A Linux distribution (or "distro") is a combination the Linux kernel with a bunch of GNU software (licensed under the GPL).  Companies combine these things together, add their own enhancements and slap on their own branding (such as Red Hat Linux or Ubuntu Linux).

MacOS X is a complete operating system descended from NextOS, which pre-dates Linux, based on BSD (Berkley) Unix with its own kernel called XNU.  XNU is descended from the Mach kernel created at Carnegie Mellon University.  As with Linux distributions, MacOS X does ship with quite a lot of open source tools, and Apple does contribute code back to the open source community.

MacOS X also, of course, has a standard GUI, which has long been one of the best and most obvious features of the Mac.  Linux has no standard GUI.  Linux users have to choose from one of at least 8 different desktop GUI environments, which is great for people who like lots of options, but not so great for people who just want something that works.

Long-time Linux users may be annoyed that MacOS X doesn't have a command-line tool for installing open source software packages and their dependency like apt-get or yum.  There are some open source projects that try to fill that need.  The most popular seems to be Homebrew.  Homebrew is pretty good for installing many software packages ported from Unix or Linux to MacOS X.

Most Mac users will probably never mess with something like Homebrew.  They will get all their software from the App Store or distributed as disc image or installer.  However, as a developer there are things I need that are easier to install via Homebrew.  It can be slow, and it does not play well with competing package manager solutions, but it gets the job done.

The reason I recently made the switch from Windows, at least for my work laptop, is that the Dell laptop they had given me crashed constantly with sudden blue screen.  The IT Help Desk couldn't fix the problem, and I got tired of dealing with it and asked them to get me a Macbook instead.  After my new Macbook was ordered I figured out that my Dell problems were caused by a bad memory card, but the Macbook was already on its way.

I bought a Macbook for my wife several years ago, because I got tired of constantly fixing the problems she had with Windows, like viruses or the dreaded "blue screen of death" that occurred seemingly at random.  The Mac turned out to be a good choice for her.  I was happy that I didn't have to help her as often, and she was happy to have something that is easy to use and just works nearly all of the time.

In the years since I bought my wife's Macbook, I went through several less expensive Windows laptops.  I tried HP, Asus, and Toshiba and unfortunately had quality issues with all three.  The reason I kept buying them hoping the next one would be better is that, of course, I am cheap.  Actually, Apple computers only seem more expensive because they do not offer a low-end product made with cheap hardware.  If Apple wanted to sell low-end hardware, of course they could.  As they say, you get what you pay for.

Apple is primarily a hardware company, which is why they hired Microsoft to create Microsoft Word for the original Mac.  They do not have a business model like Microsoft's where they expect to make a lot of money from their software or like Google's where they try to make money by giving away free services and charging for advertising.  They make money by producing innovative products that their customers are willing to pay a premium for.

Anyway, once you get used to the differences in the GUI and the keyboard layout, a Mac can do basically everything a Windows PC, and you can even run a lot of Windows software in a virtual machine or with Wine (the Windows emulator).

But you shouldn't get a Mac because it can emulate a Windows PC, but because of the reliable and enjoyable user experience.  You will never see another "blue screen of death", you will not have to worry about your computer becoming infected with viruses (thanks to XProtect and the fact that most viruses target Windows, but if you want additional protection there are several free antivirus offerings), and you will never again feel that you should keep your operating systems 2 or 3 major releases behind the latest.