Tag
13 posts
The artificial neuron model used by Jeff Hawkins and Subutai Ahmad in their new paper (image reproduced from their paper, and cropped). Their neuron model is inspired by the pyramidal cells found in neocortex layers 2/3 and 5. It has been several years since Jeff Hawkins and Numenta published the Cortical Learning Algorithm (CLA) whitepaper.…
By Gideon Kowadlo, David Rawlinson and Alan Zhang Can you hear silence or see pitch black? Should we classify no input as a valid state or ignore it? To my knowledge, the machine learning and statistics literature mainly regards an absence of input as missing data. There are several ways that it’s handled. It can…
TL;DR: An SDR is a Sparse Distributed Representation, described below SDRs are biologically plausible data structures SDRs have powerful properties SDRs have received a lot of attention recently There are a few really great new resources on the topic: Presentation by Subutai Ahmad of Numenta Older introductory presentation by Jeff Hawkins Excellent summary of their characteristics in…
This post asks some questions about the agency of hierarchical action selection. We assume various pieces of HTM / MPF canon, such as a cortical hierarchy. Agency The concept of agency has various meanings in psychology, neuroscience, artificial intelligence and philosophy. The common element is having control over a system, with varying qualifiers regarding the entities…
This post is about a fantastic new paper by Michael R. Ferrier, titled: Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function The paper was posted to the NUPIC mailing list and can be found via: http://numenta.org/community-content.html The paper itself is currently hosted at:…
It’s exciting to see growing interest and participation in the AGI community. This is another brief post to share two examples. In this case, they both build on one approach to AGI, and that is HTM. FXAI – Explorations Into Computational Intelligence A blog that is pretty well described by the title. The author, Felix…
This is a quick post to link a poster paper by Ryan McCall, who has experimented with a Predictive-Coding / Cortical Learning Algorithm (PC-CLA) hybrid approach. We found the paper via Ryan writing to the NUPIC theory mailing list. What’s great about the paper is it links to some of the PC papers we mentioned…
Browsing the NUPIC Theory mailing list, I came across a post by Fergal Byrne on the differences and similarities between Deep Learning and MPF/HTM. It’s a great background into some of the pros and cons of each. Given the popularity and demonstrated success of Deep Learning methods it’s good to understand how they work and how…
Introduction Predictive Coding (PC) is a popular theory of cortical function within the neuroscience community. There is considerable biological evidence to support the essential concepts (see e.g. “Canonical microcircuits for predictive coding” by Bastos et al). PC describes a method of encoding messages passed between processing units. Specifically, PC states that messages encode prediction failures; when…
A quick post here to give readers an idea of HTM/CLA performance on visual object recognition tasks. Although a little out of date now, Craig’s blog gives a good idea of what NUPIC can and can’t do, and how data issues (e.g. training set, variations in object appearance) affect the results. This is a handy…
by Gideon Kowadlo and David Rawlinson Introduction One of the keys to understanding the neocortex as a whole, and the emergence of intelligence, is to understand how the cortical hierarchical levels interconnect. This includes: the physical connections, the meaning of the signals being transmitted, and possibly also the way the signal is encoded. Physical connections: Physical…
by David Rawlinson and Gideon Kowadlo Article Series This is the first of 3 articles about temporal pooler design for Numenta’s Cortical Learning Algorithm (CLA) and related methods (e.g. HTM/MPF). It has limited relevance to Deep Belief Networks. This first part will describe some of the considerations, objectives and constraints on temporal pooler design. We…
by David Rawlinson and Gideon Kowadlo Introduction The Memory Prediction Framework (MPF) is a general description of a class of algorithms. Numenta’s Cortical Learning Algorithm (CLA) is a specific instance of the framework. Numenta’s Hierarchical Temporal Memory (HTM) was an earlier instance of the framework. HTM and CLA adopt different internal representations so it is…