The Transformer – now that’s an AI I think I can finally get behind, but I have to study how it works first. 29 years of looking at neural net models, unhappy with what I see (except for Danko Nikolic’s model, even though Markov is always fun). This seems close to Practopoesis’ Anapoeisis, which is to me a gold standard model of generated intelligence. It may even match up if it has sufficient similarities. Gotta read. “The Transformer (Vaswani et al., 2017), a sequence model based on self-attention, has achieved compelling results in many generation tasks that require maintaining long-range coherence” https://arxiv.org/abs/1809.04281

The Transformer – now that’s an AI I think I can finally get behind, but I have to study how it works first.
 
29 years of looking at neural net models, unhappy with what I see (except for Danko Nikolic‘s model, even though Markov is always fun).
 
This seems close to Practopoesis’ Anapoeisis, which is to me a gold standard model of generated intelligence. It may even match up if it has sufficient similarities. Gotta read.
 
“The Transformer (Vaswani et al., 2017), a sequence model based on self-attention, has achieved compelling results in many generation tasks that require maintaining long-range coherence”
 
https://arxiv.org/abs/1809.04281
===

YES! I think this is the model. It relies ENTIRELY on attention using a combination of absolute and relative spacing coordinate backgrounds.

Took you guys long enough.

https://arxiv.org/pdf/1803.02155.pdf

===

 

THIS IS IT. The RIGHT direction in Neural Network architecture 29 years, with only Danko Nikolic Practopoeisis / Anapoesis bucking the trend, I’ve looked for someone to notice the importance of “noticing” for proper Neural Network functioning.
 
While “Attention” isn’t _all_ you need, Attention is key to agility in constantly changing contexts while also maintaining an intermediate view that’s also attached to slow evolutionary processes – 3 tiers interacting continually at their respective paces.
 
This paper starts with an absolute grid. Next influential paper adds relative attention along with absolute attention.
 
The Transformer. Watch this AI.
 
I take it back. Attention _is_ all you need.
 
https://arxiv.org/abs/1706.03762
=====

 

Leave a comment

Your email address will not be published. Required fields are marked *


× five = 5

Leave a Reply