CoPilot Chat Now Available for All – Stay Informed with This Week in AI

GitHub Launches Copilot Chat for All Users, Bringing AI Assistance to Developers

GitHub has announced the open availability of Copilot Chat, an AI-powered chatbot designed to assist developers using Microsoft's IDEs, Visual Studio Code, and Visual Studio. This move follows Copilot Chat's initial rollout to Copilot for Business subscribers and later to individual Copilot customers in beta, who pay a $10 monthly fee. Now, GitHub is expanding access to all users, making it accessible in the sidebar of Microsoft's IDEs.

Shuyin Zhao, VP of product management at GitHub, expressed GitHub's commitment to providing innovative tools for developers, stating that Copilot Chat, powered by OpenAI's GPT-4, is now the most widely adopted AI developer tool in history. It is offered as part of GitHub Copilot paid tiers and is free for verified teachers, students, and maintainers of specific open-source projects.

Copilot Chat allows developers to seek real-time guidance by prompting the chatbot in natural language. Developers can ask questions, seek explanations for concepts, detect vulnerabilities, or even request code snippets. The chatbot relies on GPT-4, OpenAI's flagship generative AI model, fine-tuned specifically for developer scenarios.

However, GitHub's use of publicly available data, including copyrighted or restricted-license content, has led to legal challenges. Several class action lawsuits have been filed against GitHub, Microsoft (GitHub's parent company), and OpenAI, alleging open-source licensing and intellectual property violations. Despite concerns, GitHub has not introduced a mechanism for codebase owners to opt out of training.

Zhao suggested that codebase owners make their repositories private to avoid being included in future training sets. Given the various reasons for keeping copyrighted code public, this proposal may not sit well with codebase owners, such as crowdsourcing bug hunting.

Generative AI models, including GPT-4, are prone to hallucinations, potentially introducing buggy or deprecated code snippets. Zhao emphasized that GPT-4 performs better against hallucinations than the previous model, and Copilot Chat incorporates features like filters for insecure code patterns to mitigate vulnerabilities. However, she stressed the importance of close human review when using AI-suggested code.

GitHub faces financial challenges with Copilot, as reported losses average $20 per user per month, with some customers costing as much as $80 per month. The underlying AI models are the primary reason for these high costs. GitHub's competitor, Amazon's CodeWhisperer, has been upgraded with a free tier, a professional tier with enhanced features, and an enterprise plan. GitHub strives to make Copilot profitable amid stiff competition and financial constraints.

Copilot has seen substantial adoption despite these challenges, with 1 million paying users and approximately 37,000 enterprise clients, according to Microsoft CEO Satya Nadella. GitHub aims to continue empowering developers with AI tools and enhancing Copilot's attractiveness to maintain its competitive edge.

Giga ML Aims to Revolutionize Enterprise LLM Adoption with On-Premise Deployment

The demand for large language models (LLMs), particularly in the realm of text-generating AI, is on the rise, with 67.2% of surveyed enterprise organizations expressing the adoption of LLMs as a top priority by early 2024. However, customization, flexibility, and data privacy challenges hinder the smooth integration of LLMs into production environments, according to a recent survey.

In response to these obstacles, Varun Vummadi and Esha Manideep Dinne founded Giga ML, a startup dedicated to solving enterprise LLM adoption challenges. Giga ML's platform allows companies to deploy LLMs on-premise, offering potential benefits such as cost reduction and enhanced privacy protection.

Vummadi highlighted the significant challenges faced by enterprises in adopting LLMs, stating, "Data privacy and customizing LLMs are some of the biggest challenges faced by enterprises when adopting LLMs to solve problems." Giga ML aims to address these challenges by simplifying the deployment process and providing an easy-to-use API for training, fine-tuning, and running LLMs.

Giga ML introduces its own set of LLMs, known as the "X1 series," designed for tasks such as generating code and responding to common customer queries. Built atop Meta's Llama 2, Giga ML claims that its models outperform popular LLMs on specific benchmarks, particularly the MT-Bench test set for dialogs. While the qualitative comparison is challenging due to technical issues encountered in the online demo, Giga ML prioritizes creating tools that empower businesses to fine-tune LLMs locally without relying on external resources.

Vummadi clarified that Giga ML's mission is not necessarily to produce the best-performing LLMs but to provide tools that enable enterprises to deploy LLMs securely and efficiently on their own on-premise infrastructure or virtual private cloud. The focus is on simplifying the training, fine-tuning, and running of LLMs while ensuring data compliance.

The privacy advantages of running models offline are a vital selling point for Giga ML. Many enterprises hesitate to use commercial LLMs due to concerns about sharing sensitive or proprietary data with external vendors. Predibase's survey revealed that less than a quarter of enterprises are comfortable using commercial LLMs, citing privacy, cost, and customization issues.

Giga ML, backed by approximately $3.74 million in venture capital funding from investors like Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, and others, plans to expand its two-person team and invest in product research and development. The startup serves enterprise clients in the finance and healthcare sectors, providing them with secure on-premise deployment of LLMs, customizable models, and fast inference for enhanced data compliance and efficiency.