skyline of Raleigh, NC

Nick Freeman


This exercise carries the double duties of:

  1. offering an interesting and non-trivial use case for exploring how JavaScript proxies work, and
  2. constructing a somewhat × {elegant, partial} solution for what I personally feel to be a shortcoming in how JavaScript arrays work.

Background

What’s wrong with JavaScript arrays?

For the purposes of this article, my gripe has to do with how we can access elements relative to the end of an array. Let’s say that we have an array alphabet, such as

const alphabet = ['a', 'b', 'c', 'd', 'e', 'f', /* ... */, 'z'];

// ~or~ you could do 
const alphabet = Array.from('abcdefghijklmnopqrstuvwxyz');

If we want to get an element at a position relative to the beginning of an array, we can simply do a property lookup using an integer offset with bracket notation:

alphabet[0]; // "a"
alphabet[1]; // "b"

We always know that the first element of an array (which may be empty) is at offset 0. Of course, if we always know the length of the array ahead of time, we can just use a known highest index to get the last element.

alphabet[25]; // "z"

However, if we don’t know the number of elements in the array (its “length”) ahead of time (which would be most of the time, I would think), how do we come up with an index number to get that array’s last element at runtime?

alphabet[alphabet.length - 1]; // "z"

If you’ve worked with JavaScript for any amount of time, this is all probably quite obvious to you already, and having to do the repeated .length - 1 dance just seems normal by this point.

I’ve worked with JavaScript for some amount of time now, and this is all quite obvious to me; .length - 1 seems normal.

Me too, pal, but hear me out: In the never-ending effort to write clear and intentional code, having to repeat the name of the variable when we only care about one property lookup doesn’t seem quite right. There are two main reasons that I dislike the above syntax:

  • I believe it unnecessarily obscures the author’s intent – just wanting to get the last element doesn’t necessarily mean that we care about how long the array is.

  • It discourages the use of longer, more descriptive variable names for arrays if the author anticipates having to repeat its name more than once on one line or statement.

Potential Mechanics

.pop()

Array.prototype.pop does return the last element of an array, but popping also removes that element from the array entirely (complementing .push which adds a new element onto the end).

alphabet.pop(); // "z"
alphabet[alphabet.length - 1]; // "y" // oops!

This may be useful if we’re implementing some sort of stack or queue, but not good if we don’t want to mutate the array itself.

.slice()

Array.prototype.slice gives us a way to make a shallow copy of a certain part of an array given optional begin and end arguments. Interestingly enough, if the begin argument is a negative integer, it acts as an offset from the end of the array – which is the kind of behavior we’re looking for!

let last = alphabet.slice(-1); // ["z"]
typeof last; // "object" // oops!

This is pretty close, but still has some drawbacks. .slice always returns an array – in this case, .slice(-1) will always return an array with one element (the last element of the original array), or an empty array if the original array was itself empty. To get the element itself, we have to use one more indirect step:

let last = alphabet.slice(-1)[0]; // "z"
typeof last; // "string"

Currently, this is actually my preferred way of getting the last element in my own production code, even though it is admittedly an indirect approach.

If we want to get the nth-last element, we must also supply an exclusive end index so that .slice does not return the entire tail of the array starting at begin.

let secondLast = alphabet.slice(-2, -1)[0]; // "y"

Ew, that’s ugly

Totally agree. I would argue that the above is even less readable than alphabet[alphabet.length - 2].

.lastItem()

The Array.prototype.lastItem method is currently a stage 1 proposal, so it is possible that it (or something like it) will become standardized in the future. Its polyfill is also relatively straightforward, and so could be used even sooner.

alphabet.lastItem(); // "z"

However, the .lastItem method still has what I consider to be some odd characteristics. Invoking a method using parenthesis looks very different than accessing an element using bracket notation. It also does not offer a way to access elements at an arbitrary offset from the end of the array.

The same proposal also provides the property Array.prototype.lastIndex, which effectively returns «array».length - 1. To get the second-to-last element of our array, we could use:

alphabet[alphabet.lastIndex - 1]; // "y"

Is this more clear than alphabet[alphabet.length - 2]? Maybe, maybe not, but that may largely depend on your personal cognitive proclivities.

Desired Mechanic

I would like to be able to just use a negative integer index with bracket notation for both getting and setting elements of the array relative to its end.

// desired code usage:

alphabet[-1]; // "z"
alphabet[-2]; // "y"

alphabet[-1] = 'zed'; // "zed"

Some high-level programming languages already do this, such as Python1 and Ruby.2 There are, however, very valid reasons why adding such behavior natively is not possible in ECMAScript.

The use of bracket notation for property lookup is not unique to objects which are instances of Array. Like any other computed property lookup, the numbers in brackets are implicitly coerced to strings, as property keys can only be strings or symbols.

alphabet['0']; // "a"

It is entirely possible for someone to add arbitrary properties on their array objects, and some of those property keys may themselves be the string representation of negative numbers.

String(1.000); // "1"
String(-1); // "-1"

let someArr = ['a', 'b', 'c'];
someArr[1.000]; // "b"

someArr['-1'] = 'uhoh';
someArr[-1]; // "uhoh"

Because production code out in the wild might rely on this behavior, it cannot be changed, according to the First Law of Web Standards: thou shalt not break the web. How, then, can we alter the behavior of arrays in our own code in order to get nice, succinct negative offset indexing?

Prior Art

Both of these offer simple and similar ways to get elements from the end of an array using a proxy. However, there are, in my view, essential use case and edge case considerations for which they do not account. That is not to say that I think they are in any way bad, as they appear to be largely thought exercises just as we are doing here now. I just happen to find this topic interesting and want to throw my own ideas into the mix.

Metaprogramming and Proxies in JavaScript

Although introduced in ES2015 (ES6), Proxies are a language feature which I have thus far never used in any production application, nor have I seen them used much elsewhere in the wild. JavaScript has had similar “metaprogramming” features – various sorts of reflection and dynamic programming – available for much longer, especially since ES5 introduced property descriptors and many Object.* introspection methods.

For example, object properties can be given getters and setters which can use custom functions to intercept the lookup and setting of properties, respectively.

Using a custom property descriptor

We could use a custom getter and setter on the property with key '-1' on our array.

Object.defineProperty(alphabet, '-1', {
  get() {
    return this[this.length - 1];
  },
  set(val) {
    this[this.length - 1] = val;
  },
  // enumerable and configurable default to false
});

alphabet[-1] = 'zed'; // "zed"
alphabet[-1]; // "z"

This works fine for the single use case above, but it only affects our one existing array instance. If we wanted all instances of Array to have this behavior, we could define the property description on Array.prototype instead.

Object.defineProperty(Array.prototype, '-1', /* ... */);

However, this runs into the same dilemma that we identified before. Because this would now affect all array instances in the program – including, presumably, third-party code – this could potentially break any code which relies on the previous standard behavior, intentionally or not.

Subclassing Array with custom getter/setter

When I first played with the idea of implementing this feature, I also tried creating a subclass extending Array and similarly defining its own getter and setter for the property '-1'. Let’s call this class NegArray.

class NegArray extends Array {
  get [-1]() {
    return this[this.length - 1];
  }
  set [-1](val) {
    this[this.length - 1] = val;
  }
}

An important consequence of this is that, as MDN describes,

In ES2015, the class syntax allows for sub-classing of both built-in and user defined classes; as a result, static methods such as Array.from are “inherited” by subclasses of Array and create new instances of the subclass, not Array.

let arr = NegArray.from(alphabet);

let mappedArr = arr.filter(c => c <= 'f').map(c => c.toUpperCase());

mappedArr instanceof NegArray; // true

mappedArr[-1]; // "F"

This seems to have gotten us quite far. Once a user has an instance of our custom array class, they can use it just as they would expect to be able to use normal Array instances, including using standard methods such as .map and .filter. Even those methods which return new array instances will properly return instances of our subclass. This is incredibly important, and we will later have to take this into account with the proxy solution as well.

However, we still have not implemented all of the desired functionality. The above property definitions only work to access the last element in the array, not elements which are nth-last from the end.

What getters do not provide is the ability to intercept property lookups on arbitrary keys, without knowing the key ahead of time. Before proxies, JavaScript did not provide for this level of programmatic interception on property access, like some other languages do – PHP, for example, oddly calls it “overloading” using the magic method __get().

Setting Traps with Proxies

Proxies enable an even deeper level of metaprogramming by allowing us to “define custom behavior for fundamental operations (e.g. property lookup, assignment, enumeration, function invocation, etc),” according to MDN. I’ve included some more terminology from that page here for reference:

Terminology

handler
Placeholder object which contains traps.
traps
The methods that provide property access. This is analogous to the concept of traps in operating systems.
target
Object which the proxy virtualizes. It is often used as storage backend for the proxy. Invariants (semantics that remain unchanged) regarding object non-extensibility or non-configurable properties are verified against the target.

For our purposes, we’re interested in two fundamental operations: property lookup and assignment, which have the traps get and set, respectively. Let’s start fleshing out our proxy handler with its initial structure and some rough notes on the logic that it will have to implement:

const handler = {
  get(target, key) {
    // TODO:
    // if key is a negative index, look up element from the end
    // else, pass lookup on to underlying target
  },
  set(target, key, value) {
    // TODO:
    // if key is a negative index, set as element from the end
    // else, pass setter on to underlying target
  },
};

First, we must determine if the key being looked up is a valid negative index. Rather than putting the logic inside of the get trap, I’ll implement a little helper function to abstract the logic and hopefully make it more readable.

According to the ECMAScript spec, a valid array index is an integer i whose numeric value is +0 ≤ i < 232 − 1.

/**
 * Whether a given key is a valid negative index
 * @param key {string|symbol}
 * @return boolean
 */
function isNegIndex(key) {
  // We must check for symbol first and early-return,
  // otherwise we will get a TypeError when trying to coerce it to a number.
  if (typeof key == 'symbol') return false;

  let num = Number(key); // Coerce to number

  if (!Number.isInteger(num)) return false; // Must be integer

  if (num > 2 ** 32 - 1) return false; // Must be < 2^32 - 1

  return num < 0; // Check if negative
}

Now we can use that helper function in the handler:

const handler = {
  get(target, key) {
    if (isNegIndex(key)) {      // TODO: look up element from the end    }    // TODO: else, pass lookup on to underlying target
  },
  set(target, key, value) {
    if (isNegIndex(key)) {      // TODO: set as element from the end    }    // TODO: else, pass setter on to underlying target
  },
};

How might we easily calculate the position of an element relative to the end? Just use the underlying array’s existing .length property. If we want the nth-last element, then we just have to subtract n from length in order to find the correct 0-based index. In fact, since n was originally given as (a string representation of) a negative number, we can add that negative offset to length in order to get the correct index.

const handler = {
  get(target, key) {
    if (isNegIndex(key)) {
      return target[target.length + Number(key)];    }
    // TODO: else, pass lookup on to underlying target
  },
  set(target, key, value) {
    if (isNegIndex(key)) {
      target[target.length + Number(key)] = value;      return true;    }
    // TODO: else, pass setter on to underlying target
  },
};

To finish things off, we should transparently pass through any requests for keys that are not negative indices.

const handler = {
  get(target, key) {
    if (isNegIndex(key)) {
      return target[target.length + Number(key)];
    }

    return target[key];  },
  set(target, key, value) {
    if (isNegIndex(key)) {
      target[target.length + Number(key)] = value;
      return true;
    }

    target[key] = value;    return true;  },
};

Now that we have a handler, we can create a new proxy object that wraps an instance of an array. Remember our original alphabet array?

const alphabet = ['a', 'b', 'c', 'd', 'e', 'f', /*...*/ , 'z'];

let a_proxy = new Proxy(alphabet, handler);

a_proxy instanceof Array; // true

a_proxy[-1]; // "z"
a_proxy[-2]; // "y"
a_proxy[-3]; // "x"

So far, so good! The need to use new Proxy(...) is a bit verbose, but that seems easy to abstract. How does it hold up with standard Array methods?

let mappedArr = a_proxy.filter(c => c <= 'f').map(c => c.toUpperCase());

mappedArr[-1]; // undefined // oops! what happened?

While a_proxy is itself a proxied instance, .filter and .map are both blissfully unaware and return new instances of Array that are not proxied.


At this point we have implemented two solutions that each partially provide our desired functionality:

  • Using a proxy allows us to programmatically look up offsets other than -1, but the behavior is lost on arrays received from prototype methods.

  • Subclassing Array automatically lets Array.prototype.* methods return new instances of our custom subclass, but it is untenable to write any negative index accessors other than that for -1.

How might we combine these approaches to get the best of both worlds?

A Combined Approach

// use same `handler` object from above

class NegArray extends Array {
  constructor(...args) {
    super(...args);
    return new Proxy(this, handler);
  }
}

Wait, can it really be that simple?

Yes! But there’s a lot going on in those 6 lines, much of it condensed thanks to a generous helping of ES6 ES2015 sugar.

The extends mechanism tells the engine to take care of wiring up all of the necessary connections between various prototypes, as normal.

The constructor function that we wrote simply passes on any arguments it receives to the native Array constructor. The trick, though, is at the end of the constructor function: instead of implicitly returning the newly-created instance of NegArray, it instead returns a proxied instance.

Because this constructor will be used by all of the Array.prototype.* methods which create new instances, It Just Works™.

let n_alphabet = NegArray.from('abcdefghijklmnopqrstuvwxyz');

n_alphabet instanceof NegArray; // true
n_alphabet instanceof Array; // true

n_alphabet[-1]; // "z"
n_alphabet[-2]; // "y"
n_alphabet[-3]; // "x"

let mappedArr = n_alphabet.filter(c => c <= 'f').map(c => c.toUpperCase());

mappedArr[-1]; // "F"
mappedArr[-2]; // "E"

mappedArr[-3] = 'hello';

mappedArr.join(); // "A,B,C,hello,E,F"
  • Easy, non-verbose construction
  • Get element from arbitrary end offset
  • Set element from arbitrary end offset
  • Maintain functionality on chained instances

Handling Edge Cases

Our solution so far is well and good, but we have not yet considered what should happen if the user attempts to use a negative index with an absolute value greater than then length of the array.

let shortArr = NegArray.from(['a', 'b', 'c']);

shortArr.length; // 3
shortArr[-3]; // "a"

shortArr[-4]; // ???

shortArr[-4] = «some value»; // ???

From a perspective inside the proxy, calculating target.length + Number(key) will still end up with a negative number. If the proxy saved a passed value on a negative index of the target, it would technically work, but that value would, in many senses, be lost. Because the value was not saved as a normal element (that is, one with a positive index number), it will not be picked up by any of the standard Array functions or iterators.

// using same shortArr from above

shortArr[-4] = '@';
shortArr[-4]; // "@"

// The string "@" is actually stored in the target array as property with
// key '-1'. If the length of the array changes, the value will no longer be
// accessible using its original negative offset.

shortArr.push('d');
shortArr.length; // 4
shortArr[-4]; // "a"
shortArr[-5]; // "@" // internally: (.length - 5) = -1

shortArr.join(''); // "abcd" // element "@" is lost

Array.from(shortArr)[-1]; // undefined

While this behavior may seem logical to those of us with “insider knowledge” of what’s going on in the proxy handler, it may be unexpected and unintuitive to users of the proxied arrays. What other kind of behavior might we desire, from the perspective of a NegArray user?

Option 1: throw RangeError

// in `handler`

set(target, key, value) {
  if (isNegIndex(key)) {
    let idx = Number(key);    if (idx > target.length)      throw new RangeError(`index ${idx} is out of bounds`);    target[target.length + idx] = value;    return true;
  }

  target[key] = value;
  return true;
}
let n_arr = NegArray.from('abc');

n_arr[-4] = 'z'; // 🔺RangeError: index -4 is out of bounds

While this may in some sense follow the spirit of failing fast, in practice it would just lead users of NegArrays to have to write even more code to safeguard against RangeErrors.

let n_arr = NegArray.from(/* «something» */);
let myNum = -42;

if (Math.abs(myNum) <= n_arr.length) {
  n_arr[myNum] = 'foobar';
} else {
  // do something else I guess...
}

/* or even... */

try {
  n_arr[myNum] = 'foobar';
} catch (e) {
  if (e instanceof RangeError) {
    // now do something with this error...
  }
}

// Wait... didn't we originally start on this whole journey to make
// things easier and less verbose in the first place?

Additionally, in order to match this behavior, we would also need to throw RangeErrors when trying to read negative indices before the beginning of the array. That would work, but it is probably not as intuitive or idiomatic as simply returning undefined, which is how most of JavaScript chooses to deal with nonexistent things already.

Option 2: Shift existing array elements right

On normal arrays, setting a value at an index equal to or greater than its current length will succeed, updating the array’s length. The new “holes” up until the last element are not filled in, not even with the value undefined. They are left empty, and are not included when many Array.prototype.* methods are used.

let arr = ['a', 'b', 'c'];
arr.length; // 3

arr[25] = 'z';
arr.length; // 26

arr; // ["a", "b", "c", empty × 22, "z"]

arr.join(''); // "abcz"

When setting a value at an index “before” the beginning of the array, we could add a similar auto-expanding behavior, just at the beginning rather than the end. This requires that we add (“unshift”) the new value as the first element of the array and shift the rest of the elements to the right such that the array’s new length equals the absolute value of the given negative index. In other words, it should behave as:

let n_arr = NegArray.from(['z']);

n_arr[-26] = 'a';
n_arr[0]; // "a"
n_arr.length; // 26

n_arr; // ["a", empty × 24, "z"]

Just as setting array[+n] guarantees that array has a length of at least n+1, so too does setting array[-n] guarantee that array will have a length of at least n.

JavaScript does not natively provide an automatic method for auto-expanding an array by an arbitrary amount in its beginning in the same way that it does for an array’s end. Though it may not be the most performant method,[measurement needed] one possible solution is to use the standard Array.prototype.reverse method to temporarily reverse the underlying array in place, add the new element at the end such that the length is correctly updated and the holes are correctly placed (err– rather, left alone), then use .reverse() again to restore the correct order.

// in `handler`

set(target, key, value) {
  if (isNegIndex(key)) {
    let idx = target.length + Number(key);    if (idx >= 0) {      target[target.length + Number(key)] = value;    } else {      let shiftAmount = 0 - idx; // == Math.abs(idx) - 1      target.reverse();      target[target.length + shiftAmount] = value;      target.reverse();    }    return true;
  }
  target[key] = value;
  return true;
}

Option 3: Not support any assignment using negative indices

… and leave such behavior undefined. But that’s no fun now, is it?


Out of these, I personally think that Option 2 is a better choice given that it follows existing JavaScript usage patterns.

For the property lookup (getter), the answer to the same dilemma is relatively straightforward: if the user tries to access a negative index with a magnitude greater than the length of the array, just return undefined. This is effectively the current behavior now that, given Option 2, the underlying target array will not have any properties set that actually have negative indices.

let n_arr = NegArray.from(['a']);

n_arr[-2]; // undefined

Bailing Out

Once a user has an instance of a NegArray, how can they go about escaping back into the world of normal Arrays?

Luckily, bailing out is as easy as – and exactly mirrors – opting in:

let n_arr = NegArray.from(['a', 'b', 'c']);

let arr = Array.from(n_arr);

arr instanceof Array; // true
arr instanceof NegArray; // false

arr[-1]; // undefined

If you are writing a library or utility that uses special subclasses of standard object types, it is probably a good idea to convert them back into a standard type before returning to your own code consumers.

Filling in holes

Using the .from method does introduce one subtle change in array content when converting back and forth between Arrays and NegArrays. Remember the “holes”, or empty elements in the array that we discussed earlier? The .from method implicitly makes use of its argument’s iterator, if present. Array iterators will return undefined for empty indices instead of skipping them.

let arr = ['a', , 'c']; // ["a", empty, "c"]
1 in arr; // false

let arr2 = Array.from(arr); // ["a", undefined, "c"]
1 in arr2; // true

The difference between an empty element and an element which contains undefined is very subtle and often not of any concern given a particular use case. However, in case you do care about preserving holes in sparse arrays, we can instead implement functions to create a copy of the underlying array containing the values only of defined indices.

class NegArray extends Array {
  /* ... */

  static fromArray(arr) {
    let n_arr = new NegArray(arr.length); // correctly set the length
    for (let i in arr) {
      // only use existent, enumerable keys/indices
      n_arr[i] = arr[i];
    }
    return n_arr;
  }

  toArray() {
    let copy = new Array(this.length);
    for (let i in this) {
      copy[i] = this[i];
    }
    return copy;
  }
}

Note that our fromArray method is static, so it will be invoked as NegArray.fromArray.

let n_arr = NegArray.fromArray(['a', , 'c']);
1 in n_arr; // false

n_arr[-1]; // "c"

let arr = n_arr.toArray(); // ["a", empty, "c"]

arr instanceof Array; // true
arr instanceof NegArray; // false
1 in arr; // false
arr[-1]; // undefined

If you wanted to be more specific about the purpose of such methods, you could also name them something like fromSparseArray and toSparseArray, respectively. Depending on your desired behavior, you may also want to add additional checks to make sure that only enumerable properties are used that are valid array indices. I’ll leave this as an exercise for you, dear reader.

The Full Solution

// NegArray.js

const handler = {
  get(target, key) {
    if (isNegIndex(key)) {
      return target[target.length + Number(key)];
    }
    return target[key];
  },
  set(target, key, value) {
    if (isNegIndex(key)) {
      let idx = target.length + Number(key);
      if (idx >= 0) {
        target[idx] = value;
        return true;
      }
      let shiftAmount = 0 - idx;
      target.reverse();
      target[target.length + shiftAmount] = value;
      target.reverse();
      return true;
    }
    target[key] = value;
    return true;
  },
};

export class NegArray extends Array {
  constructor(...args) {
    super(...args);
    return new Proxy(this, handler);
  }

  static fromArray(arr) {
    let n_arr = new NegArray(arr.length);
    for (let i in arr) {
      n_arr[i] = arr[i];
    }
    return n_arr;
  }

  toArray() {
    let copy = new Array(this.length);
    for (let i in this) {
      copy[i] = this[i];
    }
    return copy;
  }
}

function isNegIndex(key) {
  if (typeof key == 'symbol') return false;
  let num = Number(key);
  if (!Number.isInteger(num)) return false;
  if (num >= 2 ** 32 - 1) return false;
  return num < 0;
}

Sweet! So how can I use this in production?

Oh goodness, please don’t. There are probably subtle and obscure native Array behaviors that I have not accounted for here, and I have not done any performance testing or tuning.

And on top of that, because negative array indexing isn’t (and likely will never be) a standardized thing in JS, other people reading your code would have no clue what was going on (or they might think that you have no clue about how JS arrays work).

If a lot of people are interested, I may consider publishing a similar implementation as a package on npm, complete with tests and typedefs and all that. Let me know what you think. Have you ever wanted negative indexing in JS for yourself? In what other use cases would you use such a package? Would you do anything differently? Did you find a mistake in any of my reasoning? Let me know!

Future Work

In addition to wanting negative indexing functionality, I have also long desired other special array (or list, matrix, etc.) shorthand to make various patterns of slicing and sequencing easier. I am inspired much by Python’s and MATLAB’s prolific use of array and matrix slicing using colons :, and Ruby’s array ranges using double-dot .. notation. It seems to me that such features could similarly be implemented in a roundabout way using proxies – of course, with quite a bit more logic and care regarding edge cases.

I will probably play around with such features in a future article.

Until then, dear reader,

… stay positive! 😉

to top