PBS 87 of X — JavaScript Iterators Objects & Generator Functions
In the previous instalment we started our redux and update of function objects in JavaScript. This fits into a larger mini-series within the larger series looking at all the different proverbial hats objects wear in JavaScript. The previous instalment was almost all redux, this one by contrast will be entirely dedicated to updated features added in ES6 that we’ve not seen before.
The instalment has a some-what strange structure because I felt it best to change my plans a little and extend the existing challenge by another week. Hopefully you won’t mind the slightly different flow of the sections compared to the norm.
You can download this instalment’s ZIP file here. If you prefer, you can access this instalment’s resources directly at the following links:
pbs87a.html: View Page or View Sourcepbs87b.html: View Page or View Source
Listen Along: CCATP Episode 615
https://media.blubrry.com/nosillacast/traffic.libsyn.com/nosillacast/CCATP_2019_11_30.mp3
PBS 85 Challenge Update
It had been my intention to include the sample solution to the challenge set at the end of instalment 85 at the start of this instalment, and to end with a new challenge, but I’ve changed my mind on that. As I was working on my sample solution two things became clear to me — the challenge was a lot more difficult that I had intended, and, a good solution will rely heavily on jQuery, so it will make the perfect hook for a redux and update of our look at DOM objects and jQuery objects.
A Top-up for Extra Credit
If you’ve already completed the challenge and are very disappointed not to be getting any new ‘homework’, I offer a little top-up to this challenge for some proverbial extra credit.
If you can, add some extra UI to allow users to decide which currencies should be shown as the rows within the cards. Remember that each card represents a currency’s rate against a list of other currencies. So far those ‘other currencies’ have been hard-coded. The primary challenge is to simply add control over the cards, not to add control over the rows within the cards, so this literally adds an extra dimension to the problem.
Iterators
As you’ve probably noticed over these past few instalments, ES6 brought absolutely massive changes to JavaScript. Iterators are yet another new feature introduced to the language in ES6. We’ve actually already used iterators, but we’ve done so implicitly rather than explicitly.
Let’s start with the problem to be solved — it is often necessary to perform an action on a series of values, i.e., it is often necessary to iterate over a series of values. ES6 added the so-called iterator protocol to provide a standard mechanism for iteration. We won’t be going into the details here, but we do need to understand the official JavaScript terminology.
An object which can be iterated over can be said to be iterable. All JavaScript Arrays are iterable, as are JavaScript strings (one character at a time).
All iterables can produce an iterator object on demand.
Iterator objects provide a .next() function for iterating over the values represented by the iterator object. Each time you call .next() a dictionary (plain object) with two keys is returned — the first key is value, and will be the current value in the sequence, and the second key is done, a boolean indicating whether or not the end of the sequence has been reached.
Note that iterator objects are designed to be used once and then discarded, they are not intended as permanent references to the underlying data. When an iterator reaches the end of its sequence it is useless, there is no .previous() function, and there is no way to re-start the sequence!
As of ES6, a number of JavaScript objects are now iterable, including arrays, strings, and the special arguments objects within functions. Iterating over arrays and arguments seems obvious, but iterating over strings may seem a little stranger at first glance. The iterator objects produced from strings iterate over the string one character at a time. The developers of third-party libraries can also implement the iterator protocol, making their objects iterable too if they so desire. Modern versions of jQuery have added iterator support, so the objects returned from the $() function are now iterable, producing iterators that iterate one tag at a time.
Iterator Object Example
To play along with these examples, open a JavaScript console on the file pbs87a.html.
This all sounds very abstract, so let’s create an iterator object and interact with its .next() function. To do this we’ll use an array, and we’ll make use of the fact that as of ES6 the Array class/prototype provides a .values() function which returns a fresh iterator object representing the array’s current values.
// create an array const myArray = [‘first val’, ‘second’, ‘3rd’];
// create an iterator object for the values in the array const myIterator = myArray.values();
// call the iterator object’s .next() function four times console.log(myIterator.next()); // {value: “first val”, done: false} console.log(myIterator.next()); // {value: “second”, done: false} console.log(myIterator.next()); // {value: “3rd”, done: false} console.log(myIterator.next()); // {value: undefined, done: true}
Iterators and the for...of Loop
To date, when describing what the for...of loop does I’ve used vague language like for...of loops can iterate over arrays and for...of loops can iterate over array-like objects. We can now be exact! Both iterables and for...of loops were introduced in ES6, and that’s no coincidence — the truth is that for...of loops can iterate over any iterable.
While you can use .next() in a while loop to iterate over an iterable, in reality you’ll usually use for...of loops.
Let’s take a moment to look at for...of iterating over things other than arrays.
Firstly, we can use for...of to iterate over the special arguments variable that exists within all functions:
// define a function that iterates over it arguments function argLister(){ console.log(`Received the following ${arguments.length} args(s)`); for(const arg of arguments){ console.log(`* ${arg}`); } }
// call the function with 2 arguments argLister(‘howdy’, ‘doody’);
// call the function with 4 arguments argLister(‘howdy’, ‘doody’, ‘boogers’, ‘snot’);
Now let’s iterate over a string:
for(const l of “boogers!”){ console.log(`l=${l}`); }
Finally, let’s demonstrate iterating over a jQuery object. The file pbs87a.html contains an un-ordered list with the ID jq_iter_demo that contains 3 list items. We can iterate over it like so:
// define an array of colour classes const colourClasses = [‘text-primary’, ‘text-success’, ‘text-danger’];
// randomly colour each list item for(const li of $(‘#jq_iter_demo li’)){ // get a random number between 0 and 2 inclusive const randIdx = Math.floor(Math.random() * 1000) % 3;
// remove all classes then add a random class $(li).removeClass().addClass(colourClasses[randIdx]); }
We need to break this very short snippet down to understand it.
The first thing to note is that the iterator objects created by jQuery iterate over native DOM objects, not jQuery objects. This is consistent with how callbacks work in jQuery, so while people may quibble with that choice, it is at least consistent. This means that to treat each iterated value as a jQuery object we have to pass it to the $() function. In the example I chose to name the values produced by the iterator li, but before I can call jQuery’s .addClass() function I have to convert it from a native DOM object to a native DOM object with $(li).
Note that '#jq_iter_demo li' is a CSS selector that selects all <li> tags within an element with the id jq_iter_demo.
Finally, also note the use of jQuery function chaining to first remove all classes with .removeClass(), and then add in the randomly chosen class.
This is a very contrived example, but it does show how you use for...of loops with jQuery, and it’s all thanks to iterators 🙂
Generator Functions
Iterable objects are very useful for storing ordered data, but there are more things you may wish to iterate over than stored data. You may wish to iterate over a series of results from some kind of calculation. This is where so-called generator functions come in.
Generator functions can be used to create so-called generator objects, and those generator objects are iterable. As well as being iterable, generator objects also have a directly accessible .next() function so there is no need for an extra step to get from an iterable to an iterator object (i.e. no equivalent to myArray.values()). In effect, a generator object does triple-duty as a generator object, an iterable, and an iterator object.
In practice, generator functions are used to create generator objects, and their values are iterated over using .next() or for...of loops.
If it helps, you can think of generator functions as iterator object factories.
Creating Generator Functions with * and yield
Before we look at the exact syntax for creating a generator function, we need to explain a new concept — resumable functions.
So far in everything we’ve seen throughout this series, functions are black boxes that take an arbitrary amount of arguments, execute a sequence of instructions in order, optionally return a single value, and end. They are completely linear, and once a function starts executing it continues until it finishes.
Generator functions are difference! They still take arbitrarily many arguments, they are still executed in order, and they can still return a single value if desired, but they can also pause execution at any time by yielding a value. Yielding is very similar to returning except that it doesn’t end the function, instead, it pauses it. A yield can emit a value, like return does, but it can also receive new arguments when execution resumes. A generator function’s scope is not destroyed when it yields, so when execution resumes all local variables will be just as they were when the function yielded.
The keyword yield is used to yield values from and to a generator function.
Generator functions are special functions, so you create them using a special syntax. The idea is similar to how async functions are defined, but a little different, instead of a keyword like async, generator functions are pre-fixed with the * character.
You can create generator functions using function statements like so:
function* myFirstGenerator(){ yield ‘boogers’; return ‘snot’; }
You can also create generator functions using function literals like so:
const mySecondGenerator = function*(){ yield ‘boogers’; return ‘snot’; };
Note that you can use loops within generator functions, and, that it is perfectly acceptable to have a generator function with an intentional infinite loop that will keep yielding values for ever. Such infinite generator functions will yield, but not return.
Using Generator Functions
Generator functions are called to create generator objects, and those objects are then interacted with to step through a sequence of values.
Let’s consider the very simple sample generator included in pbs85a.html:
function* basicGenerator(){ console.log(‘basic generator: started to execute’); yield ‘first yielded value’; console.log(‘basic generator: resumed after first yield’); yield ‘second yielded value’; console.log(‘basic generator: resumed after second yield’); return ‘final returned value’; }
This function yields two strings, then returns a string.
To use this generator function we first call it to create a generator object and store it in the global variable myGeneratorObj, then call .next() on that generator object.
// call the generator function to create a new generator object myGeneratorObj = basicGenerator();
// call next on the generator object 3 times console.log(myGeneratorObj.next()); // {value: “first yielded value”, done: false} console.log(myGeneratorObj.next()); // {value: “second yielded value”, done: false} console.log(myGeneratorObj.next()); // {value: “final returned value”, done: true}
The first thing to note here is that creating the generator object does not execute the generator object. Nothing gets logged until the first time we call .next() on the generator object.
The second thing to note is that when a value is yielded done is false, but when a value is returned, done is true.
Calling generators directly can be useful, but you can also iterate over all their values with a for...of loop. Because generator objects are also iterator objects they can be used directly within a for...if loop, but there is a caveat, for...of loops only iterate over yielded values, they ignore returned values:
// create a generator object myGeneratorObj = basicGenerator();
// iterate over it for(const val of myGeneratorObj){ console.log(`got: ${val}`); }
// logs: // —– // got: first yielded value // got: second yielded value
So, there are two things of note here. Firstly, the for...of loop automatically unpacks the dictionaries actually returned by .next(), so val contains the actual value yielded. And secondly, only the yielded values are included in the loop, the returned value is ignored. I honestly have no idea what it works like this, but it does!
Generator Function Arguments
Generator functions can accept arguments in the same way any other functions can.
Let’s illustrate this by looking at a more practical example that is designed to play nice with for...of loops. Our generator will produce a series of random numbers of a requested length. The number of random numbers desired will be passed as the first argument to the generator function. You’ll find this function defined in pbs87a.html, but I’m including the full code below:
// basic random number generator function* basicRNG(n){ while(n > 0){ yield Math.random(); n–; } }
Note that the function only uses yield, and does not use return. This is to ensure for...of loops see all the random numbers.
We can now use this generator like so:
// create a generator object for 5 random numbers myGeneratorObj = basicRNG(5);
// iterate over the random numbers and print them for(const rn of myGeneratorObj){ console.log(rn); }
There is of course no need to create a separate variable, so we can collapse this down down like so:
// iterate over 5 random numbers and print them for(const rn of basicRNG(5)){ console.log(rn); }
Infinite Generators
Our examples so far have all been finite series of values, but in reality many series go on for ever.
While infinite series make no sense in a for...of loop, they can be very useful in other contexts, particularly interactive contexts where a user can keep performing an action as often as they like.
To illustrate this point let’s look at a more real-world random number generator. Again, this function is defined in pbs87a.html, but included below for convenience.
This better random number generator will be able to provider either a finite or infinite number of random numbers, depending on how it’s called. If called with no arguments it will produce an infinite stream, if called with a number as the first argument it will produce that many random numbers.
// better random number generator function* rng(n=0){ if(n > 0){ // generate a finite number of random numbers while(n > 0){ yield Math.random(); n–; } }else{ // generate an infinite stram of random numbes while(true) yield Math.random(); } }
This improved generator can still be used within for...of loops to generate finite series:
// iterate over 3 random numbers and print them for(const rn of rng(3)){ console.log(rn); }
But it can now also be used with .next() to generate infinite series:
// create a new generator object for an infinite series myGeneratorObj = rng(); // no args
// call .next(as often as desired) console.log(myGeneratorObj.next());
To make this example a little more real-world, pbs87a.html contains a web UI for generating random numbers using this generator. The HTML markup is quite straight forward:
Random Number Generator
This is basically just a Bootstap card containing a bootstrap form consisting of a button to generate a random number, and a text box to show it. The key points to note are that the button has the ID rng_btn, and the text box has the ID rng_tb.
This UI is brought to life in the document ready handler by creating an infinite RNG generator object and adding a click handler to the button:
// create an infinite RNG generator object const rngObj = rng();
// add a click handler to the randon number button $(‘#rng_btn’).click(function(){ $(‘#rng_tb’).val(rngObj.next().value).select(); });
As you can see, the code to bring this form to life is very short. The key things to note are that rngObj.next().value will always evaluate to the next random number, and the jQuery .val() and .select() functions are used to set the content of the text box and then select. The reason I chose to select the newly generated number is to make it easier for users to copy-and-paste the number somewhere.
Passing Values Back to yield
I mentioned previously that yield can receive as well as emit values. This might seem like a strange concept at first — return is very much a one-way street after all! But, since the yield keyword is used to pause and resume the function, it does actually make sense. When yield triggers a pause can emit a value, and when execution is resumed it can receive a value.
The data flow to and from yield statements is handled by the .next() function.
I like to think of the yield keyword and the .next() function as two end of a wormhole — when you yield a value it gets returned by .next(), and when you pass a value to .next() it gets emitted by yield.
This might all sound a little confusing, so let’s look at a practical example. We can use a generator function to implement an accumulator — a counter that can be incremented by an arbitrary amount, and can be asked for its current value at any time.
Again, you’ll find the code in pbs87a.html, but I’m also including it here:
// an acumulator implemented with a generator function function* accumulator(initVal){ // if an initial value was passed, store it // otherwise start at zero const initValNum = Number(initVal); // force to number let balance = initValNum ? initValNum : 0;
// keep updating the balance for ever while(true){ // yield the current balance and accept an increment const incBy = yield balance; const incByNum = Number(incBy); // force to number
// if a valid increment was passed to next(), apply it
if(incByNum){
balance += incByNum;
} } }
The important thing to note here is the use of the value potentially emitted by yield:
const incBy = yield balance;
The value to the right of the yield keyword will be returned by .next(), and the yield operation will evaluate to value passed to .next(). In this case that means that the value passed to .next() will get stored in the variable incBy.
To use this generator function we first create a generator object with it, and we can then read out the current value by calling .next() with no arguments, and update the value by calling .next() with a number as the only argument.
Because a generator function does not even start to execute until the first time .next() is called, the first call .next() cannot be connected to a yield statement, so, any value passed to it will vanish into the ether. For this reason we need to prime the pump on our accumulator by calling .next() once after we create it:
// create a fresh accumulator accumulatorGenObj = accumulator(); accumulatorGenObj.next(); // step the generator forward to the first yield
// read out the current value console.log(accumulatorGenObj.next().value);
// increment the value by 10 and read out the new total console.log(accumulatorGenObj.next(10).value);
// increment the value by 2 and read out the new total console.log(accumulatorGenObj.next(2).value);
// read out the total one last time console.log(accumulatorGenObj.next().value);
As with the random number generator example, it’s easy to build a simple UI for an accumulator using jQuery and Bootstrap. You’ll find such a UI in pbs87a.html.
The markup for the form is very simple: