(All using LLaVa 13B v0, defaults of 0.7 temp, 512 output tokens)

I put in these three images (warning! First one is a big spider!) https://imgur.com/a/PgQT202

For the spider I asked “What kind of spider is this? Is it dangerous?”

It replied “The image features a large spider on a small bricks on a wall, with a close-up view of the spider in the center of the frame. It is a creepy crawly spider, as some might describe it. The spider appears to be walking on or sitting on the bricks, and seems to be doing some exploring outside. While the presence of a large spider may not be everyone's favorite, it is a part of the natural world and contributes to the diversity of species in the image taken with a Nikon Coolpix, capturing the small details of the spider and its surroundings.”

Critique: I believe it’s a wolf spider, though I would also believe trapdoor spider and maybe black house spider. Initially I thought it was a funnel-web spider, which is why I took a picture. The general Australian wisdom would be “looks similar enough to a funnel-web, treat it as very dangerous, just in case”. Taken with a zoomed in iPhone, not a Nikon Coolpix. It is obviously not exploring outside.

For the mushroom I asked “What is this?”

It replied “The image shows a close-up view of a rock on the ground, surrounded by grass and other plants. It appears to be on a rocky path with some leafy plants nearby. This scene might be in a natural outdoor setting or even a garden.”

Critique: it’s a big solitary mushroom, at night, lit by a powerful torch. The surroundings are correctly identified (gravel path and lightly hardened natural outdoor setting).

For the screenshot I asked “What’s so funny about this screenshot?”

It replied “In the image, there are two posts on a social media platform, both containing text and appearing as screenshots of text messages. These posts seem to be part of a conversation between two people. One post is in a foreign language, and the other is in English. The latter is a response to the former, possibly discussing the content of the initial message.”

Critique: not a conversation, latter is not a response to the former, they are merely both discussing the same topic. Neither post is in a foreign language. No attempt to address why it might be funny. (I would accept any attempt even if it’s wrong. Something like “one post says something won’t happen, while right next to it, there is another post reporting it has happened” would get partial credit. Full credit would have to acknowledge a few of the incongruities: People worried about “runaway agi” are concerned about catastrophic risk, but it is intentionally misunderstood to mean eloping with the AGI as a romantic partner, rendering the reassurance worthless; the claim that it has already happened likewise appears to be misunderstanding as well since it is reported in the manner of a pet running away, a somewhat different scale of tragedy and incongruous because we don’t think of AGI as a pet.)

Overall not super impressed with the quality so far. Describing images is impressive for sure, but subtleties, humour, and being able to draw on common wisdom are some of the main appeals of “multi-modal GPT“ and this is lacking that spirit.