Introducing GenAI studio

Today we are excited to announce the beta release of our newest product, GenAI studio: an out-of-the-box, playground experience that enables developers of all skill levels to utilize the power of generative AI, available in Determined Enterprise (MLDE).

Why GenAI studio?

GenAI studio is a new feature set of Determined that gives users an easier entrypoint to test AI foundation models for their use cases. Here are a few key benefits:

• Out-of-the-box playground: Quickly test out open-source foundation models on your domain data in the no-code GUI environment or a Jupyter notebook.

• Easy ramp-up from prototypes to production: Smoothly transition from foundation model selection and prompt engineering to fine-tuning and deployment.

• Security-first environment: GenAI studio works on any infrastructure – on-prem hardware or the cloud. That way, you can keep your data private for a unique “security-first” experience, and self-host your own model.

What can you do with GenAI studio?

1. Seamless Prompt Engineering

Select your favorite open-source foundation model from Llama2 , Falcon, or MPT, craft prompts with instructions, examples, and input, and then test them out immediately. Save your preferred configurations (including model, prompt, and optional dataset) as ‘Snapshots’ so you can preserve your work and reproduce it later. A prompt has the following parts:

• Instruction: Specific guidelines for how your model should behave. This helps the model understand user intentions better.
• Examples (Optional): Example input-response pairs so your LLM can pattern match what the outputs should look like. Give as few (one example for one-shot prompting) or as many (few-shot prompting) as you prefer.
• Input: The actual question you want to ask your LLM.
• Expected Output (Optional): Ground-truth answer for your input, in case you want to run evaluation or finetuning on your Snapshot.

For example, let’s say you want to design a prompt for customer complaint classification:

2. Scale Up Generation on Your Team’s Data

Want to scale up your prompt design to more than just one example? Load up your custom dataset, or drop a link to a Hugging Face dataset. Check out batch generations on the “consumer_complaints_short” dataset, using Llama2 7B and the same prompt we designed in step 1. Scaling up your prompt to apply to a whole dataset generates a column of responses:

You can iterate on these prompt designs and save as many Snapshots as you like. If prompt engineering alone is not getting you the results you need, you can jump into fine-tuning using your Snapshots.

3. Simple, One-click Fine-tuning

Kick off the Fine-tuning wizard to easily fine-tune a model based on your favorite Snapshot. You can also customize any of the prompt settings, or model hyperparameters further using the no-code interface before launching your training job. Either way, no need to worry about costly mistakes from manual infrastructure setup 😎

4. Easy Integration with Determined

As a GenAI studio user, you also get the full suite of enterprise-grade ML training benefits from the Determined platform. The jobs you launch from GenAI studio show up in Determined. Check them out in your Web UI to monitor performance metrics as you finetune your model:

5. Deploy Fine-tuned models to Hugging Face without writing code

Once you are satisfied with the results of your fine-tuning, load your model to your ‘Playground’ and chat with it, or deploy it to Hugging Face without writing any code. This way others can benefit from your GenAI studio development work and use your models in their applications.

6. Easy-to-Use, Customizable Notebook Experience

For GenAI studio users who want even more customizability in their use cases (or just love Python!) the Jupyter Lab tab takes you straight to a notebook environment. This is perfect for doing things like:

• Running evaluation (BLEU, ROUGE scores, or something custom using EleutherAI’s LM evaluation harness)
• Having more fine-grained control over your prompt techniques, snapshots, training loops, and allocated resources to run the job
• Further customizing your dataset using advanced data parsing
• Creating data visualizations and comparisons based on your model results

How to get started with GenAI studio

We’re excited to hear from you! If you are interested in early access to the beta product, sign up here.

Check out the beta documentation for more information.

Stay up to date by joining our Slack Community!