NVLM: Open Frontier-Class Multimodal LLMs
Paper
•
2409.11402
•
Published
•
72
Muhammad Rizki Al-Fathir
PetaniHandal
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
about 2 months ago
NVLM: Open Frontier-Class Multimodal LLMs
Reacted to
singhsidhukuldeep's
post
with 👍
about 2 months ago
Researchers have introduced OpenDevin, an open-source platform for building powerful AI agents that interact with the world through software interfaces.
Here is a speed-run of features:
- Flexible agent abstraction, allowing easy implementation of diverse AI agents
- Sandboxed Linux environment and web browser for safe code execution and web interaction
- Core actions including IPythonRunCellAction, CmdRunAction, and BrowserInteractiveAction
- AgentSkills library with reusable tools like file-editing utilities and multi-modal document parsing
- Multi-agent delegation for complex task solving
- Comprehensive evaluation framework with 15 benchmarks across software engineering and the web
Here is how you get Devin working:
1. Set up the environment:
- Install OpenDevin by following the instructions in the GitHub repository (https://github.com/OpenDevin/OpenDevin).
- Ensure you have the necessary dependencies installed.
2. Choose an agent:
- Select an agent from the AgentHub, such as the CodeActAgent or BrowsingAgent.
- Alternatively, create your own agent by implementing the agent abstraction.
3. Configure the environment:
- Set up the sandboxed Linux environment and web browser.
- Mount any necessary files or directories into the workspace.
4. Define the task:
- Specify the task you want the agent to perform, such as writing code, debugging, or web browsing.
5. Initialize the agent:
- Create an instance of your chosen agent.
- Set any necessary parameters or prompts.
6. Start the interaction:
- Begin the agent's execution loop, which typically involves:
a. The agent perceiving the current state
b. Deciding on an action
c. Executing the action in the environment
d. Observing the results
Continued in comments...
Organizations
None yet