Matrix Orthogonalization Improves Memory in Recurrent Models

(ayushtambde.com)

53 points | by at2005 6 hours ago

2 comments

  • BirbSingularity 5 hours ago
    I can't help but think of orthogonal frequency-division multiplexing and it's use in encoding data on multiple carrier frequencies, and it makes me wonder what other parallels we will discover between digital transmission technology for cross-domain stuff like this.
    • dapperdrake 5 hours ago
      Not even cross-domain. (Nor cross-co-domain.)

      Trigonometric polynomials are also polynomials. And linear spaces are all "the same". That is what the definition is for. Even the transpose-mapping is linear.

    • chimpanzee2 47 minutes ago
      I have this strange sensation that I can't put into words that somehow we are on the brink of unveiling an entirely new paradigm of AIs or perhaps even of combining AI with classical algorithms in a way to rapidly iterate between each other (and sensor data) that will instantly 10x or 100x current capabilities.

      Anyone else feel this?

      • cyanydeez 43 minutes ago
        no. we're approach a sigmoid. AI is bloated carcass and we're tweaking out the size of the models and speed they'll run on smaller hardware.

        I think to feel what you're feeling, you've bought into "all we need is more context". I think evolution demonstrates that's not really true.

        • chimpanzee2 35 minutes ago
          would you really bet that this is it? there is nothing beyond this?

          reminds me of the famous anecdote of a 19th century physics professor who said "there is nothing left to be discovered in physics, only minor corrections"

          then came Einstein...

  • mv_d5339e31 3 hours ago
    [dead]