NNUEs are an edit made to Multi-Layer Perceptrons that makes them very efficient and usable as an evaluation function for chess engines, NNUEs are quite recent (introduced in 2018) and using them alongside 0x88 and MTD(f) is quite funny if you ask me! basically honda civic with a jet engine basically
MLPs? Backpropagation?
Multi-Layer Perceptrons is usually the first thing one gets introduced to in deep learning, it's the most fundamental NN one can get exposed to after the Perceptron, i suggest watching 3b1b's really awesome deep learning series! before continue-ing on reading this article!
NNUE?!
NNUEs are just MLPs, but edited in a way that is efficient for chess engines, for our version we'll only have one hidden layer, keep in mind that engines like Stockfish uses 3 hidden layers, but the implementation details of Stockfish's NNUE are far more than what im willing to spend time comprehending :P
Since we're trying to make an NNU(Efficient) the input layer
The first optimization we can use are accumulators, which are basically an optimization on the first hidden layer, basically instead of doing the 786x128 operations each time, we can take advantage of the fact that the difference in number of changed squares after a move in chess is kinda small, so we add/remove that instead in each changed position!
// we update all the accumulators.
void add(int ind) {
for(int i = 0 ; i < 128 ; i++) acc[i] += hidden_w[i][ind];
}
void rem(int ind) {
for(int i = 0 ; i < 128 ; i++) acc[i] += hidden_w[i][ind];
}we also update in our make/unmake function in alpha-beta.
NOTICE THAT WE NEED TWO ACCUMULATORS, since our MLP only was trained on only white's position, so we need to make a copy where black is white and where white is... white, so that we can have correct evaluation for both in the NNUE.
[to be continued]