ART-1 neural network

How ART-1 works

ART-1 is an adaptive resonance network for binary vectors, described by Gail Carpenter and Stephen Grossberg in 1987. It sorts patterns into categories on its own, with no teacher and no correct answers. Each category lives in a neuron of the recognition layer as two weight vectors: B, the bottom-up weights the neuron uses to respond to a pattern, and T, the top-down weights that form the binary prototype of the category.

Patterns are presented one at a time, and each goes through three steps.

1. Recognition. Every neuron computes S = Σ b_i·x_i. The neuron with the largest S wins.
2. Comparison. The winner sends back its prototype T, and the comparison layer keeps only the shared ones: c_i = x_i·t_i. If the match ratio of X and C is at least the vigilance ρ, the network resonates. If it is lower, a reset disables the winner until the end of this pattern.
3. Learning. On resonance the winner learns: T = C, so its prototype shrinks to the shared ones. If no neuron fits, the pattern is stored in a new neuron.

New categories come from the uncommitted neuron: all its t_i are 1 and all its b_i are equally small. If it responds more strongly than the committed neurons, the pattern becomes a new category. This is how the network learns new things without erasing old ones.

Formulas

• Uncommitted neuron: t_{u i} = 1, b_{u i} = b_u < L / (L − 1 + m), where m is the pattern size.
• Recognition: S_k = Σ b_ki·x_i; for the uncommitted neuron S_u = b_u·Σx_i.
• Comparison: c_i = x_i·t_ki.
• Match ratio: r = k/m, where k is the number of positions where X and C agree, or r = |C|/|X|, the number of ones in C divided by the number of ones in X.
• The winner learns: t_ki = c_i, b_ki = L·c_i / (L − 1 + Σc).
• New neuron: t_i = x_i, b_i = L·x_i / (L − 1 + Σx).

L is a constant greater than 1, usually 2 or 3. The higher the vigilance ρ, the stricter the comparison and the more categories you get.

What you can change

• Match ratio. By default r = k/m, the share of matching positions, zeros included. Classic ART-1 uses r = |C|/|X|. On the same patterns the two can give a different number of categories.
• What happens after a reset. By default the winner is disabled and the next largest S wins; if that is the uncommitted neuron, a new category is created. The other option creates a new neuron right after the first reset, a simplification some courses use.
• b_u. If the field is empty, the solver takes the largest fraction 1/n below L/(L − 1 + m). With m = 10 and L = 2 the bound is 2/11, so b_u = 1/6.
• Ties. When two S values are equal, the lower neuron number wins, and a committed neuron goes before the uncommitted one. The solution marks such places.

Fractions are exact: 2/7 stays 2/7, so the vigilance test and the choice of the winner never depend on rounding.

Example

Five 2×5 patterns, coded row by row from left to right: X1 = 11011 10101, X2 = 00100 11111, X3 = 11111 10101, X4 = 00100 01111, X5 = 10100 11111. Vigilance ρ = 0.85, L = 2, b_u = 1/6.

X1 and X2 become categories 1 and 2. X3 responds most strongly to neuron 1 (S₁ = 7/4), the match ratio is 9/10 = 0.9 — resonance, and the prototype stays the same. X4 joins category 2 with r = 1, and that prototype shrinks to X4. X5 first picks neuron 2 (S₂ = 5/3), but the match ratio of 0.8 is below the vigilance, so the network resets. Next comes the uncommitted neuron (S_u = 7/6), and X5 becomes category 3. The Example button loads this task.