Had a predominately designed focused stream. Really struggled with color selection. It's getting better though. I had a hard stop at an hour for the evening but still managed to make decent progress. I'm looking forward to bringing in more state management via redux eventually and really breaking the Observables and state up into manageable chunks.

One thing I also need to sort out is learning how/when events are emitted when combining observables. It appeared that zip seemed to act as and AND gate that only emited after a new value from each stream was emitted. This wasn't the desired behavior I was looking for so I might have to seek some alternatives for collapsing Observable values.

I decided to shelve the machine learning bit while I went back and read up more on the fundamentals. I feel like I wasn't able to train a model because of some basic concept I was missing.

In light of that, I decided to try and put a bit of polish on the project and start stretching some design muscles. I figured an hour would be enough time but it ended up taking more. Which is no big deal, but always a good exercise to anaylyze why things took longer than you thought they would. In this instance I ended up getting hung out on how to store the UI state. I wanted something that wasn't hackey, but couldn't think of an elegant solution without bringing a full blown library in. Which just reinforces my feelings that state is hard, and it should be treated with respect.

Had a wonderful time streaming this time around. Though I didn't achieve my goal of being able to predict future values with a Linear Regression, I learned a TON.

I decided to check out Tensorflow.js. I was really happy with it! I have previously used Apache Spark so I'm not unfamiliar with big data frameworks. Tensorflow was just simple and to the point. There didn't seem to be alot of fluff in the API and I really appreciate that style of directness.

I wasn't able to accomplish my goal because frankly I have the wrong approach. Training a Tensorflow model seems to be a use case of having a complicated model and a long training time. Since I'm trying to train a model in "realtime" I basically tripped over Gradient Descent/Learning Rates.

So what next? I am going to try to have each subsequent training pick up where the last training left off. My hope here is that even though the first set of values will have a woefully inaccurate model, eventually we will hit a steady state and be able to train a new model quickly enough(I've picked < 1s for no particular reason).

If this approach doesn't work then I'm going to look into training the model Async and have the latest value take the latest model that is trained.

This could be an interesting enough approach in it's own right as it would be nice to have longer training times for the model whenever there is a gap in stream values. I might end up trying this approach reguardless.

Another Tuesday another successful stream! I did a little prep work that I mention below for this one and it was nice having notes to follow to keep things organized. However the more I am working through my analysis the more I am questioning my approach. I think next week I am going to build a live chart of the values to get a sense of where I am and where I should god. Then after that ... Linear Regressions!

I got a little excited about the idea and ended up doing an impromptu stream ... that didn't go very well but was extremely instructive:

I spent quite a bit of time trying to create a 'last 10 values' stream. I went through several approaches and ended up with using scan. Originally I tried bufferCount however buffer only emited 'last 10' values for every 10 values. This wouldn't work as I wanted to emit a new 'last 10' for each new value. Then I attempted to use window which almost seemed to work. I was looking to create an Observable<Observable<Array<Number>>> and window provided a Observable<Observable<T>> output, but shamefully I couldn't get the behavior I was looking for.

Having thoroughly overthought the problem, a google search suggested scan and I thought 'sure why not'. It ended up doing the trick quite nicely.

Pay attention to what are Operators and what are Observables. Especially when using mergeMap/concatMap/switchMap. I regularly was getting my types wrong on my inputs and outputs.

Code: https://github.com/hankyates/websocket-stream/tree/2018-07-31-post-stream

I am trying a new thing. I am going to try and live stream coding sessions of when I am learning new things. My hope is that maybe people will engage or find these musings useful. I know that there are many programmers that I wish I could just see how they work or approach things.

I set out with a pretty simple and achievable goal. Just to get an Observable stream from a websocket. Most of the evening was spent getting stumped by finding a decent API and figuring out where things were in the rxjs namespace. At the end I was able to setup two different streams that filtered based on different values which was my goal.

I am really excited about the next phase which will be acting on these values in some sort of meaningful way.

This was backed by ElasticSearch and I stopped paying for my cluster(they're expensive). So the data no longer works for the POC below.

Proof of concept: http://brfss-map.hankyates.com/
Code: https://github.com/hankyates/louisiana-map

As an exercise I wanted to build a Chloropleth map. Well I didn't exactly start at wanting to build a chloropleth map. It started as a simple question, which Parishes in Louisiana could use more resources in improving the overall health of their population? I wanted to see which parishes had the most smokers. Which had the most drinkers. Which had the most people who smoked AND drank.

The CDC runs a telephone survey called the Behavioral Risk Factor Surveillance System and publishes the results of the data. After quite a bit of normalization of the data (which could be it's own blog post), I managed to mash up a quick proof of concept.

It's clunky. I think the data is wrong. The javascript is completely unmaintainable, but I shipped something. That alone is enough to celebrate and announce.

-Hank

My Impressions on Elm

Pros

• No runtime errors.
• Not alot of boilerplate.
• Strong types.

Cons

• Not newbie friendly at all.
• Docs can be out of date and confusing (Signals, Effects)
• Simple Debugging is difficult.

A big selling point on Elm is 'no runtime errors' and this was true and amazing. If it compiled it worked. The tradeoff is that instead of debugging errors in dev tools, you spent more time figuring out what the compiler error was trying to tell you. The compilers errors are helpful though.

There wasn't a need for linting either. The types kept you in line and made sure you weren't doing anything silly. I really enjoyed writing signatures for my functions and then figuring out the implementation later. It makes designing nice API's easier.

Simple debugging was way too difficult. My Json Decoder wasn't working properly, and I had no real way of knowing that was where the problem was, because there is no simple way to console.error error messages when they occur. I eventually got it all wired up, but if I were a newbie coming in, I don't think I would have been able to get past this problem. Googling questions and error messages didnt always yield great results either. For figuring out how to Json Decode a JS Object into a List, I found an old blog post using out of date Elm API's. I eventually got what I needed from it, but considerable updates to the blog code needed to be made.

All in all though I really loved working with Elm and I think it's great for lightweight projects! I still think it doesn't quite have the community support or ecosystem to try and bring it in for a major project at work.

The problem with currying is I understand the concept. I understand what it is doing and what problems it solves. But where does one actually use it in real world code. I've been on the lookout at work and trying to shoehorn it in wherever possible but it hasn't quite fit until recently.

function sendRequest(res, template, context) {  
  if (!res.finished) {
      res.send(template(context)));
  }
}

This was the first chunk of code where the light finally turned on for me.

I love having the params passed in to this function. It makes this function pure and easy to write a unit test against. The problem was that it needed to be used in differents contexts where we only have 1 or 2 of the arguments. THIS is where currying comes in handy. Here is some context. This code is a serverside expressjs app.

Here is the work we do to get out index.html from the filesystem:

fs.readFile('index.html', 'utf-8', (err, file) => {  
  var template = _.template(indexHtml);
  // template is available
});

Here is a service call we make to fetch articles for our page:

articleService.getLatestArticles()  
  .then((context) => {
    //context is available
  }).catch(e => {console.error(e)})

Here is our express route:

app.get('/articles', (req, res) => {  
  //req is available
});

All of this might be overwhelming and ... it is. This is alot of context to juggle. Luckily we have a couple tricks to clean it up. First lets make a Promise for the filesystem read

function getIndex() {  
  return new Promise(resolve, reject){
    fs.readFile('index.html', 'utf-8', (err, file) => {
        if (err) {reject(err)}
          resolve(file);
    });
  }
}

Now lets use Promise#all so we fire off our calls at the same time but and resolve once both are complete.

var serviceCalls = [  
  articleService.getLatestArticles(),
  getIndex()
];

Promise.all(serviceCalls)  
  .then(([context, index]) => {
      // Now we have both our context and index.
  }).catch(e => {console.error(e)})

It's important to understand that these service calls are fetching their data at the same time and we only resolve once both have been completed. The following code executes one call after another which is much slower:

  articleService.getLatestArticles()
      .then(getIndex)
    .then(([context, index]) => {
        // context is `undefined` unless you plumb it through.
        // This is a sloppy pattern. Try to avoid using it and use Promise#all wherever you can.
    });

Anyways... So now we have our data and we need to send it back to the client. Let's put it all together:

function sendRequest(res, template, context) {  
  if (!res.finished) {
      res.send(template(context)));
  }
}

app.get('/articles', (req, res) => {  
  var serviceCalls = [
    articleService.getLatestArticles,
    getIndex
  ];

  Promise.all(serviceCalls)
    .then(([context, index]) => {
      sendRequest(res, _.template(index), context);
    }).catch(e => {console.error(e)})

});

This code works fine, so where does currying come in? It doesn't really solve any problems in this code but now that we understand the above problem lets play with currying a bit.

function sendRequestNotCurried(res, template, context) {  
  if (!res.finished) {
      res.send(template(context)));
  }
}

var sendRequest = _.curry(sendRequestNotCurried);

app.get('/articles', (req, res) => {  
  // Now that we have our req. Let's make a helper so we don't have to worry about it anymore.
  var wantsTemplateAndContext = sendRequest(req);
  var serviceCalls = [
    articleService.getLatestArticles,
    getIndex
  ];

  // Promise.all rejects all promises if one fails. We are using Q#allSettled so we can still send back an imcomplete page to the client without sending a 500 error.
  Q.allSettled(serviceCalls)
    .then(([contextResult, indexResult]) => {
        var index = indexResult.value;
        // Ignoring error handling for if the index.html fetch fails, let's create a function that can send a response to the client that takes different contexts.
        var wantsContext = wantsTemplateAndContext(index);
        var context;
        if (contextResult.state === 'fulfilled') {
          context = contextResult.value;
        } else {
          context = {errorMessage: 'Oops!'};
        }
        //Now we send out response back to the client. All arguments are satisfied now and `sendRequestNotCurried` will be called.
        wantsContext(context);
    });

});

This wasn't the greatest example of a good spot to use currying. The reason why I used it though is that in a real application, passing a function around to different parts of the program that just want a single argument is very useful. Currying is great for making functions that take many arguments easier to reason about. Also it makes the functions easier to test.

I've always heard Currying described as "A function that returns a function until all of its arguments are satisfied." It's commonly used to make utility functions. The classic example is always some adder function that has you'd never see in actual code. Below is an example of a very simple Curry that we used to help keep all of our verbs on our service class consistent.

var verb = v => function(options) {  
  var {command, data, successCallback, failCallback, timeout} = options;
  this.request({
    verb: v,
    command: command,
    data: data,
    successCallback: successCallback,
    failCallback: failCallback,
    timeout: timeout
  });
};

class Service {  
  get: verb('GET');
  put: verb('PUT');
  post: verb('POST');
  delete: verb('DELETE');
  request(options) {
    //$.ajax or http or superagent calls made here
  }
}

The confusing part of the above code that made my head hurt at first is v => function(options) {. At first glance you think it's just another anonymous function. It's not though. Arrow functions have an implicit return so this actually returns a function defition.

It's no secret that the JavaScript ecosystem has a high level churn. New libraries and frameworks coming out and pushing the boundaries is a good thing. The primary complaint seems to be the level of noise is too high. It's overwhelming for new comers who just want to build a damn app.

And that's fine.

The fundamental problem isn't the noise, it's investing in the noise. You won't get passed up if you don't pay attention to the noise. You will get passed up if you don't filter the noise, and focus on what is actually important and invest in that.

EcmaScript 6(ES6) is the future of JavaScript. Any library or framework you will use 5 years from now will be built on top of it. Coding in ES6 is significantly different from ES5. There are a ton of new features to learn, and you'll do yourself a favor by being ahead of the curve whenever libraries and frameworks start adopting new patterns that make use of the new features.

Don't get overwhelmed and use all the new features at once. Start with some of the simple features like template literals and arrow functions. Learn when it's appropriate to use them and when it's not. I've been using Tracuer (an ES6 transpiler) in production for six months and I still haven't used Generators or Promises. WeakMaps and Sets made their first appearance just this week.

Check out the es6 shim and try including it in your next project.

Questions? Comments? Hit me up on Twitter!

This is where renderComponentToString comes to the rescue:

replaceMacro: (htmlStr) => {  
  var galleryRegex = /\[Gallery:\s?([\s\d,])\]/i;
  var ids = htmlStr.match(htmlStr)[1].split(',').map((id) => parseInt(id, 10));
  var renderedGallery = React.renderComponentToString(<ImageGallery imageIds=ids/>);
  return htmlStr.replace(galleryRegex, renderedGallery)
};

I don't know if the event handlers get bound in this case. We use a jQuery gallery which get initialized after the gallery is in the DOM, So I suspect that might be saving us from having the events not getting bound.

The React documentation reccomends using renderComponentToString on the server probably for SEO purposes, however as you can see it can be handy in different use cases than that!

The answer to all these questions is "Yes."

I think it's impossible not to overwhelm Javascript students, because Javascript is overwhelming. At the CodeFellows Javascript Foundations II night course I've been teaching the last couple of weeks I threw a difficult homework assignment at my students. Everyone, including myself, paniced. I left the class feeling anxious that I might have gone "too hard." Then the Pull Requests started coming in and ... They looked great! I threw 4/5 new concepts at the class and it took some time for the students to understand in their own way, but at the end of the day most of the class had working code.

Overall the experience was definitely challenging and rewarding for both the students and myself. Often times with programming it's easy to look at everything you don't know. Looking at what we covered in the short four week course, It was clear that we had moved through a ton of material and seeing the journey from start to end was amazing!

Not to say that React doesn't have it's weaknesses. I'm not a huge fan of precompiling and all the headaches that come along with it. I think that's just the way the world is though on the Front End right now and you just have to deal with it. React isn't nearly as enjoyable without JSX.

The main advantage of React is that it is simple.

/** @jsx React.DOM */

var LikeButton = React.createClass({  
  getInitialState: function() {
    return {liked: false};
  },
  handleClick: function(event) {
    this.setState({liked: !this.state.liked});
  },
  // render is the only required method
  render: function() {
    var text = this.state.liked ? 'like' : 'unlike';
    return (
      <p className='{this.props.color}' onClick={this.handleClick}>
        You {text} this. Click to toggle.
      </p>
    );
  }
});

// This is one way to bind your component to the dom. 
// It's more common to include other components within a component though. 
React.renderComponent(  
  <LikeButton />,
  document.getElementById('example')
);

Your components have properties and state. Everything else you can handle on your own with your own tools. The React website makes mention that they essentially just want to be the V in the MVC. I think that attitude of being small and modular is what makes React so wonderful.

We have been able to do some pretty cool stuff with react using JSDom and Browserify. I hope I get to write more about it soon. I'm specifically trying to figure out an effective way to unit test React components. If you have some advice, hit me up on twitter please!

First issue is how do I make the text readable on top of an image? I'm using blur in my prototype but I still think its unreadable. I tinkered with adding a text-stroke to the heading, but I still want it to be readable for older browsers without css3 features. I'm last resorting to adding an opacity layer in front of the image. Too much opacity on a page can really hurt performance, especially with as many css3 transitions as I plan to have. I've also tinkered with adjusting the grayscale, but that didn't help readability either. Have a suggestion or an article? Hit me up on Twitter!

The second issue is the responsive layout. I would like to have the cards expand to the center of the page. The problem is when you increase the width of an element it always takes up the space to the right. I'm exploring having a negative margin-left to offset but negative margins are always a code smell so hopefully I can avoid that!

Also pen and paper for layout design is the best!

So the solution here was to do some branch gymnastics.

• git checkout -b build
• Using the fantastic grunt-gh-pages plugin we tell it to use master instead:

     'gh-pages': {
        options: {
          base: 'site',
          branch: 'master'
        },
        src: ['**']
      }

but we get errors:

hank@hank1:~/sites/hankyates.github.io$ grunt gh-pages  
Running "gh-pages:src" (gh-pages) task  
Cloning git@github.com:hankyates/hankyates.github.io.git into .grunt/grunt-gh-pages/gh-pages/src  
Cleaning  
Fetching origin  
Checking out origin/master  
Removing files  
Copying files  
Adding all  
Committing  
Pushing  
Warning: To git@github.com:hankyates/hankyates.github.io.git  
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'git@github.com:hankyates/hankyates.github.io.git'  
To prevent you from losing history, non-fast-forward updates were rejected  
Merge the remote changes (e.g. 'git pull') before pushing again.  See the  
'Note about fast-forwards' section of 'git push --help' for details.  
 Use --force to continue.

Oh no! git doesn't like that we are trying to push an orphaned branch with no history to a branch with history. Normally this is a good thing. Git has prevented us from clobbering our history, but we still have our history and our code but it's on the build branch instead now.

It's at this point we run the super dangerous command git push origin :master which deletes the master branch on github. Note that the grunt plugin clones the repository to a temporary folder, copies the compiled static site, commits, and pushes to master. So we still have a copy of master locally and can push it back if need be.

After running grunt gh-pages our site is is compiled and pushed to Github. A nice trick is to set the default branch on Github to build.