1. Sure they could
2. I imagine you'd have lots of situations where the probability of concurring, even with truthful responses, would be close to zero, so this wouldn't be a useful metric. Questions like "name some exotic birds that are edible, but not commonly eaten" could have thousands of valid answers, and so we wouldn't expect truthful responses to concur. Even for simpler questions, concurrence likely won't be verbatim, so how to you calculate whether or not responses have concurred? You need to train another model for that presumably, and that model will have some nonzero error rate, etc., etc.

what do you mean by generalizing here? Reconstruction of OOD data? Ironically, VAEs are pissing everybody because they reconstruct OOD data too well. In fact, one of the things people are dying to get to work is anomaly detection or OOD detection, but VAEs suck at it despite all attempts. Like your dog who cannot guard your house because he really likes strangers, VAEs suck at OOD detection because they reconstruct OOD too well.

Say "incredible" one more time

https://crfm.stanford.edu/2023/03/13/alpaca.html:

> We emphasize that Alpaca is intended only for academic research and any commercial use is prohibited. There are three factors in this decision: First, Alpaca is based on LLaMA, which has a non-commercial license, so we necessarily inherit this decision. Second, the instruction data is based OpenAI’s text-davinci-003, whose terms of use prohibit developing models that compete with OpenAI. Finally, we have not designed adequate safety measures, so Alpaca is not ready to be deployed for general use.

Why only academic research and not industry research? I don't see where that limitation comes from in their 3 factors.

Unfortunately I think, at least for now, that's just the way it is. This is why I personally focus on hardware architectures / acceleration for machine learning and biologically plausible deep learning. Ideas tend to matter more than compute resources in these domains.

Depends on the model. Some have difficulty even with full 8bit quantization; others you can go to 4bit relatively easily. There is some research that suggests 3bit might be the useful limit, with rarely certain 2bit models.

The consistent with the world type stuff could be built into the prompt engineering (e.g., tell the user about a map you have) and I think you could largely minimize hallucination but still have very realistic conversations

This makes me depressed because I’ve been working with the llama-index project and I feel like these huge companies are going to take my ball away 😢. They just have too many resources to build stuff.

What would be the way to detect any malicious nodes by machine learning? Let's say, I have datasets of RF signals of three kinds of drones. But my target is to detect any malicious drone except the drones I possess. I have two ideas: one is to use label two drones as good and the remaining one as malicious and my othe idea is to use unsupervised learning. Is there any better way?

Honestly I don't care if there's not complete consistency with the game world. Having it would be great, but you could do a "good enough" job with simple backstories getting prepended into the context window.

> #Alpacas #PetSounds #Sustainability

I too use hashtags in my reddit posts #better-than-gpt #ai #winning

> performs as well as text-davinci-003

No it doesn't! The researchers don't claim that either, they claim "often behaves similarly to text-davinci-003" which is much more believable. I've seen a lot of people claiming things like this with little evidence. We need some people evaluating these claims objectively. Can someone start a third party model review site?

If you actually want the NPCs to meaningfully add to the game rather than merely being mouthpieces then your approach won't work. How do you ensure what they say is consistent with the game world? E.g. what if they make up the location of a hidden treasure, offer to give you an item, etc. All of that needs to be accounted for in the game logic as well, otherwise they'll say things that make no sense in the game world.

It's actually a challenging problem and requires research. As far as I know there a very few people actively researching this area; if they are, then they certainly aren't publishing it. Hopefully my next paper which investigates using LLMs in Disco Elysium will help change that.

I ditched lightning after a while because I spent more too much time fixing it than doing productive stuff. But I regularly use wandb to track experiments and it’s mostly great!

Give every NPC a name and short background description. IE - something like the rules that define ChatGPT or Sydney, but only to give each character a backstory and personality traits. Every time you interact with one of these NPCs, you load this background description into the start of the context window. At the end of each interaction, you save the interaction to their background description so future interactions can reference past interactions. You could keep all the NPC's backgrounds in a hashtable or something with the keys being their names, and the values being their background description. That way you only need one LLM running for all NPCs.

I’ve successfully run the 13B parameter version of Llama on my 2080TI (11GB of VRAM) in 4-bit mode and performance was pretty good.

This is (somewhat) how the brain works; language and knowledge/reasoning are in separate structures and you can lose one without the other.

This is exactly what I have been looking for this days

Not sure we're there yet, but I have some active research in this area right now.

This will be huge for video games. The ability to run local inferencing on normal gaming hardware will mean every NPC can now be a smart character. I cant wait to be playing GTA6 and come across DAN walking down the streets of Vice City.

This is incredibly informative.

inference code is, the model weights are under a separate non-commercial license

It does actually send a WebSocket request to join the queue and waits for it to complete.

I thought llama was GPL licensed? Which isn’t ideal either but better than “research only”

> Vanilla autoencoders don't generalize well, but variational autoencoders have a much better structured latent space and generalize much better.

For toy problems yes, but not generally. For a generalizing Image Autoencoder, check for example ConvNextv2: https://arxiv.org/pdf/2301.00808.pdf

As a side note: The VQ-VAE from the blog post has actually really little to do with variational inference. You have basically no prior at all (uniform over all discrete latents) therefore the KL-divergence term can also be dropped. It's basically just a glorified quantized Autoencoder that could be interpreted in the language of variational models.