2024년 6월 13일 AI 소식

Summary

애플은 WWDC 2024 에서 새로운 개인 인텔리전스 시스템인 Apple Intelligence를 발표했습니다. 이는 iOS 18, iPadOS 18, 및 macOS Sequoia에 통합되어 사용자의 일상 작업을 지원하는 여러 생성 모델로 구성됩니다. OpenAI와 애플은 ChatGPT를 Apple의 다양한 플랫폼에 통합하는 파트너십을 발표했습니다. Stability AI는 텍스트-이미지 생성 모델인 Stable Diffusion 3 Medium을 출시하였으며, 이는 높은 품질의 이미지를 생성할 수 있는 소형 모델입니다. 또한 Anthropic은 AI 시스템의 안전성과 보안을 강화하기 위한 레드 팀 활동의 도전 과제에 대해 논의했습니다. Google Research는 개인 건강 및 웰빙 인사이트를 제공하는 AI 모델을 발표했으며, RecurrentGemma 모델의 새로운 버전을 출시하여 더 효율적인 심층 학습을 가능하게 했습니다.

Apple Intelligence 발표

애플 개인 인텔리전스 시스템 발표
링크, 2024년 6월 10일, Apple

• Apple Intelligence는 iOS 18, iPadOS 18, macOS Sequoia에 통합된 개인 인텔리전스 시스템임
• 여러 생성 모델로 구성되어 있으며, 사용자의 일상 작업을 지원함
• 텍스트 작성 및 수정, 알림 우선순위 지정 및 요약, 대화용 이미지 생성, 앱 간 상호작용 단순화 등의 기능 제공
• 30억 개 매개변수의 온디바이스 언어 모델과 더 큰 서버 기반 언어 모델 포함
• 모델은 사용자의 개인정보를 보호하며, 프라이버시 중심의 인프라를 사용함
• 책임 있는 AI 개발을 위한 원칙 설정: 사용자에게 지능형 도구 제공, 사용자 대표, 신중한 설계, 프라이버시 보호
• 모델은 고성능, 신속성, 전력 효율성을 보장하도록 설계됨

OpenAI와 Apple의 파트너십 발표

ChatGPT의 Apple 경험 통합
링크, 2024년 6월 10일, OpenAI

• ChatGPT가 iOS, iPadOS, macOS에 통합될 예정임
• Siri가 ChatGPT의 지능을 활용하여 사용자에게 더 나은 답변 제공
• ChatGPT는 시스템 전반의 작성 도구에서 사용할 수 있으며, 사용자 데이터는 저장되지 않음
• 사용자는 무료로 ChatGPT에 접근할 수 있으며, 구독자는 추가 기능 사용 가능
• 애플과의 협력을 통해 AI 기술의 접근성을 높이고 사용자에게 더 나은 경험 제공 목표

Stable Diffusion 3 Medium 발표

최신 텍스트-이미지 생성 모델 출시
링크, 2024년 6월 12일, Stability AI

• Stable Diffusion 3 Medium은 고품질 이미지 생성이 가능한 소형 모델임
• 소비자 PC 및 노트북에서도 실행 가능하며, 다양한 스타일의 고품질 출력 제공
• 모델은 오픈 라이선스와 크리에이터 라이선스로 제공됨
• NVIDIA 및 AMD와 협력하여 성능 최적화
• API와 디스코드를 통해 사용 가능
• 안정성과 책임 있는 AI 관행을 보장하기 위해 광범위한 내부 및 외부 테스트 실시

AI 시스템의 레드 팀 활동 도전 과제

AI 시스템의 안전성과 보안을 위한 테스트 방법
링크, 2024년 6월 13일, Anthropic

• 레드 팀 활동은 AI 시스템의 잠재적 취약점을 식별하기 위한 중요한 도구임
• 도메인별 전문가, 멀티모달, 군중 기반 레드 팀 활동 등 다양한 방법론 소개
• 체계적인 레드 팀 활동을 위한 표준화된 절차 필요성 강조
• 다양한 레드 팀 활동 방법을 통해 얻은 경험과 도전을 공유
• 정책 입안자 및 조직에 대한 권장 조치 제안

개인 건강 및 웰빙 인사이트 제공 AI

개인 건강 질문 및 데이터에 대한 이해와 추론
링크, 2024년 6월 11일, Google Research

• 새로운 대형 언어 모델을 통해 개인 건강 질문 및 데이터를 이해하고 추론함
• 모바일 및 웨어러블 장치 데이터를 활용하여 개인화된 건강 모니터링 및 추천 제공
• 수면 개선을 위한 맞춤형 권장사항 등 복잡한 건강 관련 질문에 대한 답변 제공
• Gemini 모델의 멀티모달 및 장기 컨텍스트 추론 능력 활용

RecurrentGemma 모델 출시

효율적인 심층 학습을 위한 새로운 모델
링크, 2024년 6월 11일, Google

• RecurrentGemma 모델은 9억 개의 매개변수를 가진 효율적인 심층 학습 모델임
• 더 적은 메모리 요구사항과 빠른 샘플링 속도를 제공
• 질문 응답, 요약, 추론 등 다양한 텍스트 생성 작업에 적합함
• Gemma 1 모델과 유사한 성능을 제공하며, 특히 긴 시퀀스 또는 대규모 배치 처리 시 효율성 증대
  Sources

  This GPT assists users by creating a detailed daily newspaper in Korean based on provided links. It follows these steps: read the content, summarize each content with detailed points, and write a report. The report format is:

(today’s date in 년 월 일) AI 소식,

Summary

(overall short summary, make summary with good details. for Summary section, explain the details starting with company name, e.g. OpenAI에서는 ~~~를 발표하였습니다.)

Title,

한글제목
링크, date,
company name

• detailed summary1, (개조식 문체 사용)
• detailed summary2, (개조식 문체 사용)
  …
• detailed summary N, (개조식 문체 사용)

Title,

한글제목
링크, date,
company name

• detailed summary1, (개조식 문체 사용)
• detailed summary2, (개조식 문체 사용)
  …
• detailed summary N, (개조식 문체 사용)
  …

  ###
  
  https://machinelearning.apple.com/research/introducing-apple-foundation-models
  
  Featured Highlight
  Introducing Apple’s On-Device and Server Foundation Models
  June 10, 2024
  
  At the 2024 Worldwide Developers Conference, we introduced Apple Intelligence, a personal intelligence system integrated deeply into iOS 18, iPadOS 18, and macOS Sequoia.
  
  Apple Intelligence is comprised of multiple highly-capable generative models that are specialized for our users’ everyday tasks, and can adapt on the fly for their current activity. The foundation models built into Apple Intelligence have been fine-tuned for user experiences such as writing and refining text, prioritizing and summarizing notifications, creating playful images for conversations with family and friends, and taking in-app actions to simplify interactions across apps.
  
  In the following overview, we will detail how two of these models — a ~3 billion parameter on-device language model, and a larger server-based language model available with Private Cloud Compute and running on Apple silicon servers — have been built and adapted to perform specialized tasks efficiently, accurately, and responsibly. These two foundation models are part of a larger family of generative models created by Apple to support users and developers; this includes a coding model to build intelligence into Xcode, as well as a diffusion model to help users express themselves visually, for example, in the Messages app. We look forward to sharing more information soon on this broader set of models.
  
  Our Focus on Responsible AI Development
  Apple Intelligence is designed with our core values at every step and built on a foundation of groundbreaking privacy innovations.
  
  Additionally, we have created a set of Responsible AI principles to guide how we develop AI tools, as well as the models that underpin them:
  
  Empower users with intelligent tools: We identify areas where AI can be used responsibly to create tools for addressing specific user needs. We respect how our users choose to use these tools to accomplish their goals.
  Represent our users: We build deeply personal products with the goal of representing users around the globe authentically. We work continuously to avoid perpetuating stereotypes and systemic biases across our AI tools and models.
  Design with care: We take precautions at every stage of our process, including design, model training, feature development, and quality evaluation to identify how our AI tools may be misused or lead to potential harm. We will continuously and proactively improve our AI tools with the help of user feedback.
  Protect privacy: We protect our users' privacy with powerful on-device processing and groundbreaking infrastructure like Private Cloud Compute. We do not use our users' private personal data or user interactions when training our foundation models.
  These principles are reflected throughout the architecture that enables Apple Intelligence, connects features and tools with specialized models, and scans inputs and outputs to provide each feature with the information needed to function responsibly.
  
  In the remainder of this overview, we provide details on decisions such as: how we develop models that are highly capable, fast, and power-efficient; how we approach training these models; how our adapters are fine-tuned for specific user needs; and how we evaluate model performance for both helpfulness and unintended harm.
  
  Modeling overview
  Figure 1: Modeling overview for the Apple foundation models.
  Pre-Training
  Our foundation models are trained on Apple's AXLearn framework, an open-source project we released in 2023. It builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs. We used a combination of data parallelism, tensor parallelism, sequence parallelism, and Fully Sharded Data Parallel (FSDP) to scale training along multiple dimensions such as data, model, and sequence length.
  
  We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control.
  
  We never use our users’ private personal data or user interactions when training our foundation models, and we apply filters to remove personally identifiable information like social security and credit card numbers that are publicly available on the Internet. We also filter profanity and other low-quality content to prevent its inclusion in the training corpus. In addition to filtering, we perform data extraction, deduplication, and the application of a model-based classifier to identify high quality documents.
  
  Post-Training
  We find that data quality is essential to model success, so we utilize a hybrid data strategy in our training pipeline, incorporating both human-annotated and synthetic data, and conduct thorough data curation and filtering procedures. We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator. We find that these two algorithms lead to significant improvement in the model’s instruction-following quality.
  
  Optimization
  In addition to ensuring our generative models are highly capable, we have used a range of innovative techniques to optimize them on-device and on our private cloud for speed and efficiency. We have applied an extensive set of optimizations for both first token and extended token inference performance.
  
  Both the on-device and server models use grouped-query-attention. We use shared input and output vocab embedding tables to reduce memory requirements and inference cost. These shared embedding tensors are mapped without duplications. The on-device model uses a vocab size of 49K, while the server model uses a vocab size of 100K, which includes additional language and technical tokens.
  
  For on-device inference, we use low-bit palletization, a critical optimization technique that achieves the necessary memory, power, and performance requirements. To maintain model quality, we developed a new framework using LoRA adapters that incorporates a mixed 2-bit and 4-bit configuration strategy — averaging 3.5 bits-per-weight — to achieve the same accuracy as the uncompressed models.
  
  Additionally, we use an interactive model latency and power analysis tool, Talaria, to better guide the bit rate selection for each operation. We also utilize activation quantization and embedding quantization, and have developed an approach to enable efficient Key-Value (KV) cache update on our neural engines.
  
  With this set of optimizations, on iPhone 15 Pro we are able to reach time-to-first-token latency of about 0.6 millisecond per prompt token, and a generation rate of 30 tokens per second. Notably, this performance is attained before employing token speculation techniques, from which we see further enhancement on the token generation rate.
  
  Model Adaptation
  Our foundation models are fine-tuned for users’ everyday activities, and can dynamically specialize themselves on-the-fly for the task at hand. We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks. For our models we adapt the attention matrices, the attention projection matrix, and the fully connected layers in the point-wise feedforward networks for a suitable set of the decoding layers of the transformer architecture.
  
  By fine-tuning only the adapter layers, the original parameters of the base pre-trained model remain unchanged, preserving the general knowledge of the model while tailoring the adapter layers to support specific tasks.
  
  Figure 2: Adapters are small collections of model weights that are overlaid onto the common base foundation model. They can be dynamically loaded and swapped — giving the foundation model the ability to specialize itself on-the-fly for the task at hand. Apple Intelligence includes a broad set of adapters, each fine-tuned for a specific feature. It’s an efficient way to scale the capabilities of our foundation model.
  We represent the values of the adapter parameters using 16 bits, and for the ~3 billion parameter on-device model, the parameters for a rank 16 adapter typically require 10s of megabytes. The adapter models can be dynamically loaded, temporarily cached in memory, and swapped — giving our foundation model the ability to specialize itself on the fly for the task at hand while efficiently managing memory and guaranteeing the operating system's responsiveness.
  
  To facilitate the training of the adapters, we created an efficient infrastructure that allows us to rapidly retrain, test, and deploy adapters when either the base model or the training data gets updated. The adapter parameters are initialized using the accuracy-recovery adapter introduced in the Optimization section.
  
  Performance and Evaluation
  Our focus is on delivering generative models that can enable users to communicate, work, express themselves, and get things done across their Apple products. When benchmarking our models, we focus on human evaluation as we find that these results are highly correlated to user experience in our products. We conducted performance evaluations on both feature-specific adapters and the foundation models.
  
  To illustrate our approach, we look at how we evaluated our adapter for summarization. As product requirements for summaries of emails and notifications differ in subtle but important ways, we fine-tune accuracy-recovery low-rank (LoRA) adapters on top of the palletized model to meet these specific requirements. Our training data is based on synthetic summaries generated from bigger server models, filtered by a rejection sampling strategy that keeps only the high quality summaries.
  
  To evaluate the product-specific summarization, we use a set of 750 responses carefully sampled for each use case. These evaluation datasets emphasize a diverse set of inputs that our product features are likely to face in production, and include a stratified mixture of single and stacked documents of varying content types and lengths. As product features, it was important to evaluate performance against datasets that are representative of real use cases. We find that our models with adapters generate better summaries than a comparable model.
  
  As part of responsible development, we identified and evaluated specific risks inherent to summarization. For example, summaries occasionally remove important nuance or other details in ways that are undesirable. However, we found that the summarization adapter did not amplify sensitive content in over 99% of targeted adversarial examples. We continue to adversarially probe to identify unknown harms and expand our evaluations to help guide further improvements.
  
  Figure 3: Ratio of "good" and "poor" responses for two summarization use cases relative to all responses. Summaries are classified as "good", "neutral", "poor" given the grader's scores across five dimensions. A result is classified as "good" if all of the dimensions are good (higher is better). A result is classified as "poor" if any of the dimensions are poor (lower is better). Our models with adapters generate better summaries than a comparable model.
  In addition to evaluating feature specific performance powered by foundation models and adapters, we evaluate both the on-device and server-based models’ general capabilities. We utilize a comprehensive evaluation set of real-world prompts to test the general model capabilities. These prompts are diverse across different difficulty levels and cover major categories such as brainstorming, classification, closed question answering, coding, extraction, mathematical reasoning, open question answering, rewriting, safety, summarization, and writing.
  
  We compare our models with both open-source models (Phi-3, Gemma, Mistral, DBRX) and commercial models of comparable size (GPT-3.5-Turbo, GPT-4-Turbo)1. We find that our models are preferred by human graders over most comparable competitor models. On this benchmark, our on-device model, with ~3B parameters, outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B. Our server model compares favorably to DBRX-Instruct, Mixtral-8x22B, and GPT-3.5-Turbo while being highly efficient.
  
  Figure 4: Fraction of preferred responses in side-by-side evaluation of Apple's foundation model against comparable models. We find that our models are preferred by human graders.
  We use a set of diverse adversarial prompts to test the model performance on harmful content, sensitive topics, and factuality. We measure the violation rates of each model as evaluated by human graders on this evaluation set, with a lower number being desirable. Both the on-device and server models are robust when faced with adversarial prompts, achieving violation rates lower than open-source and commercial models.
  
  Figure 5: Fraction of violating responses for harmful content, sensitive topics, and factuality (lower is better). Our models are robust when faced with adversarial prompts.
  Our models are preferred by human graders as safe and helpful over competitor models for these prompts. However, considering the broad capabilities of large language models, we understand the limitation of our safety benchmark. We are actively conducting both manual and automatic red-teaming with internal and external teams to continue evaluating our models' safety.
  
  Human Preference Evaluation on Safety Prompts
  Figure 6: Fraction of preferred responses in side-by-side evaluation of Apple's foundation model against comparable models on safety prompts. Human graders found our responses safer and more helpful.
  To further evaluate our models, we use the Instruction-Following Eval (IFEval) benchmark to compare their instruction-following capabilities with models of comparable size. The results suggest that both our on-device and server model follow detailed instructions better than the open-source and commercial models of comparable size.
  
  Figure 7: Instruction-following capability (measured with IFEval) for Apple's foundation models and models of comparable size (higher is better).
  We evaluate our models’ writing ability on our internal summarization and composition benchmarks, consisting of a variety of writing instructions. These results do not refer to our feature-specific adapter for summarization (seen in Figure 3), nor do we have an adapter focused on composition.
  
  Figure 8: Writing ability on internal summarization and composition benchmarks (higher is better).
  Conclusion
  The Apple foundation models and adapters introduced at WWDC24 underlie Apple Intelligence, the new personal intelligence system that is integrated deeply into iPhone, iPad, and Mac, and enables powerful capabilities across language, images, actions, and personal context. Our models have been created with the purpose of helping users do everyday activities across their Apple products, and developed responsibly at every stage and guided by Apple’s core values. We look forward to sharing more information soon on our broader family of generative models, including language, diffusion, and coding models.
  
  Footnotes
  [1] We compared against the following model versions: gpt-3.5-turbo-0125, gpt-4-0125-preview, Phi-3-mini-4k-instruct, Mistral-7B-Instruct-v0.2, Mixtral-8x22B-Instruct-v0.1, Gemma-1.1-2B, and Gemma-1.1-7B. The open-source and Apple models are evaluated in bfloat16 precision.
  
  ###
  
  https://openai.com/index/openai-and-apple-announce-partnership/
  June 10, 2024
  
  OpenAI and Apple announce partnership to integrate ChatGPT into Apple experiences
  Coming to iOS, iPadOS, and macOS later this year.
  
  Editor’s Note: This news was shared at Apple's Worldwide Developer Conference 2024. You can also read about the news here(opens in a new window).
  
  Apple is integrating ChatGPT into experiences within iOS, iPadOS, and macOS, allowing users to access ChatGPT’s capabilities—including image and document understanding—without needing to jump between tools.
  
  Siri can also tap into ChatGPT’s intelligence when helpful. Apple users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.
  
  Additionally, ChatGPT will be available in Apple’s systemwide Writing Tools, to help users generate content for anything they are writing about. Users can also tap into ChatGPT image tools to generate images in a wide variety of styles to complement what they are writing.
  
  Privacy protections are built in when accessing ChatGPT within Siri and Writing Tools—requests are not stored by OpenAI, and users’ IP addresses are obscured. Users can also choose to connect their ChatGPT account, which means their data preferences will apply under ChatGPT’s policies.
  
  The ChatGPT integration, powered by GPT-4o, will come to iOS, iPadOS, and macOS later this year. Users can access it for free without creating an account, and ChatGPT subscribers can connect their accounts and access paid features right from these experiences.
  
  We're excited to partner with Apple to bring ChatGPT to their users in a new way. Apple shares our commitment to safety and innovation, and this partnership aligns with OpenAI's mission to make advanced AI accessible to everyone. Together with Apple, we're making it easier for people to benefit from what AI can offer.
  Sam Altman, CEO of OpenAI
  
  ###
  
  https://stability.ai/news/stable-diffusion-3-medium
  Announcing the Open Release of Stable Diffusion 3 Medium, Our Most Sophisticated Image Generation Model to Date
  12 Jun
  Key Takeaways
  
  Stable Diffusion 3 Medium is Stability AI’s most advanced text-to-image open model yet.
  
  The small size of this model makes it perfect for running on consumer PCs and laptops as well as enterprise-tier GPUs. It is suitably sized to become the next standard in text-to-image models.
  
  The weights are now available under an open non-commercial license and a low-cost Creator License. For large-scale commercial use, please contact us for licensing details.
  
  To try Stable Diffusion 3 models, try using the API on the Stability Platform, sign up for a free three-day trial on Stable Assistant, and try Stable Artisan via Discord.
  
  Play
  00:00
  00:21
  Mute
  Settings
  Enter fullscreen
  
  00:00
  We are excited to announce the launch of Stable Diffusion 3 Medium, the latest and most advanced text-to-image AI model in our Stable Diffusion 3 series. Released today, Stable Diffusion 3 Medium represents a major milestone in the evolution of generative AI, continuing our commitment to democratising this powerful technology.
  
  What Makes SD3 Medium Stand Out?
  
  SD3 Medium is a 2 billion parameter SD3 model that offers some notable features:
  
  Overall Quality and Photorealism: Delivers images with exceptional detail, color, and lighting, enabling photorealistic outputs as well as high-quality outputs in flexible styles. Success in addressing common pitfalls of other models, such as realism in hands and faces, is achieved through innovations such as the 16-channel VAE.
  
  Prompt Understanding: Comprehends long and complex prompts involving spatial reasoning, compositional elements, actions, and styles. By utilizing all three text encoders or a combination, users can trade off performance for efficiency.
  
  Typography: Achieves unprecedented text quality with fewer errors in spelling, kerning, letter forming, and spacing by leveraging our Diffusion Transformer architecture.
  
  Resource-efficient: Ideal for running on standard consumer GPUs without performance degradation, thanks to its low VRAM footprint.
  
  Fine-Tuning: Capable of absorbing nuanced details from small datasets, making it perfect for customisation.
  
  View fullsize
  
  Our collaboration with NVIDIA
  
  We collaborated with NVIDIA to enhance the performance of all Stable Diffusion models, including Stable Diffusion 3 Medium, by leveraging NVIDIA® RTX™ GPUs and TensorRT™. The TensorRT- optimised versions will provide best-in-class performance, yielding a 50% increase in performance.
  
  Download the TensorRT-optimised version of Stable Diffusion 3 Medium.
  
  Our collaboration with AMD
  
  AMD has optimized inference for SD3 Medium for various AMD devices including AMD’s latest APUs, consumer GPUs and MI-300X Enterprise GPUs.
  
  Open and Accessible
  
  Our commitment to open generative AI remains unwavering. Stable Diffusion 3 Medium is released under the Stability Non-Commercial Research Community License. We encourage professional artists, designers, developers, and AI enthusiasts to use our new Creator License for commercial purposes. For large-scale commercial use, please contact us for licensing details.
  
  Try Stable Diffusion 3 via our API and Applications
  
  Alongside the open release, Stable Diffusion 3 Medium is available on our API powered by Fireworks AI. Other versions of Stable Diffusion 3 such as the SD3 Large model and SD3 Ultra are also available to try on our friendly chatbot, Stable Assistant and on Discord via Stable Artisan. Get started with a three-day free trial.
  
  How to Get Started
  
  Download the weights of Stable Diffusion 3 Medium
  
  For Commercial Inquiries: Contact us for licensing details.
  
  Start a three-day trial of our image services on Stable Assistant and Stable Artisan.
  
  Learn more about the model in our detailed FAQs.
  
  Safety
  
  We believe in safe, responsible AI practices. This means we have taken and continue to take reasonable steps to prevent the misuse of Stable Diffusion 3 Medium by bad actors. Safety starts when we begin training our model and continues throughout testing, evaluation, and deployment. We have conducted extensive internal and external testing of this model and have developed and implemented numerous safeguards to prevent harms.
  
  By continually collaborating with researchers, experts, and our community, we expect to innovate further with integrity as we continue to improve the model. For more information about our approach to Safety please visit our Stable Safety page.
  
  Licensing
  
  Stable Diffusion 3 Medium is released under the Stability Non-Commercial Research Community License.
  
  We have introduced the new Creator License to encourage the community to leverage Stable Diffusion 3 while supporting Stability in the mission to keep AI open and accessible. We encourage professional artists, designers, developers, and AI enthusiasts to use the Creator License to start building with Stable Diffusion.
  
  Large-scale commercial users and enterprises are requested to contact us and obtain an Enterprise License. This will help us ensure that businesses can leverage the full potential of our model while adhering to our usage guidelines.
  
  ###
  
  https://www.anthropic.com/news/challenges-in-red-teaming-ai-systems
  anthrophic
  
  Challenges in red teaming AI systems
  2024년 6월 13일
  ●
  10 min read
  Computer chip and a flame
  In this post we detail insights from a sample of red teaming approaches that we’ve used to test our AI systems. Through this practice, we’ve begun to gather empirical data about the appropriate tool to reach for in a given situation, and the associated benefits and challenges with each approach. We hope this post is helpful for other companies trying to red team their AI systems, policymakers curious about how red teaming works in practice, and organizations that want to red team AI technology.
  
  What is red teaming?
  Red teaming is a critical tool for improving the safety and security of AI systems. It involves adversarially testing a technological system to identify potential vulnerabilities. Today, researchers and AI developers employ a wide range of red teaming techniques to test their AI systems, each with its own advantages and disadvantages.
  
  The lack of standardized practices for AI red teaming further complicates the situation. Developers might use different techniques to assess the same type of threat model, and even when they use the same technique, the way they go about red teaming might look quite different in practice. This inconsistency makes it challenging to objectively compare the relative safety of different AI systems.
  
  To address this, the AI field needs established practices and standards for systematic red teaming. We believe it is important to do this work now so organizations are prepared to manage today’s risks and mitigate future threats when models significantly increase their capabilities. In an effort to contribute to this goal, we share an overview of some of the red teaming methods we have explored, and demonstrate how they can be integrated into an iterative process from qualitative red teaming to the development of automated evaluations. We close with a set of recommended actions policymakers can take to foster a strong AI testing ecosystem.
  
  Red teaming methods this post covers:
  Domain-specific, expert red teaming
  
  Trust & Safety: Policy Vulnerability Testing
  National security: Frontier threats red teaming
  Region-specific: Multilingual and multicultural red teaming
  Using language models to red team
  
  Automated red teaming
  Red teaming in new modalities
  
  Multimodal red teaming
  Open-ended, general red teaming
  
  Crowdsourced red teaming for general harms
  Community-based red teaming for general risks and system limitations
  In the following sections, we will cover each of these red teaming methods, examining the unique advantages and the challenges they present (some of the benefits and challenges we outline may be applicable across red teaming methods).
  
  ###
  
  https://research.google/blog/advancing-personal-health-and-wellness-insights-with-ai/
  google research
  Blog
  Advancing personal health and wellness insights with AI
  June 11, 2024
  
  Shwetak Patel, Distinguished Engineer & Health Technologies Lead, Google, and Shravya Shetty, Principal Engineer, Google Research
  
  Our research introduces a novel large language model that aims to understand and reason about personal health questions and data. To systematically evaluate our model, we curate a set of three benchmark datasets that test expert domain knowledge, alignment with patient reported outcomes, and the ability to produce human quality recommendations.
  
  Mobile and wearable devices can provide continuous, granular, and longitudinal data on an individual’s physiological state and behaviors. Examples include step counts, raw sensor measurements such as heart rate variability, sleep duration, and more. Individuals can use these data for personal health monitoring as well as to motivate healthy behavior. This represents an exciting area in which generative AI models can be used to provide additional personalized insights and recommendations to an individual to help them reach their health goals. To do so, however, models must be able to reason about personal health data comprising complex time series and sporadic information (like workout logs), contextualize these data using relevant personal health domain knowledge, and produce personalized interpretations and recommendations grounded in an individual’s health context.
  
  Consider a common health query, “How can I get better sleep?” Though a seemingly straightforward question, arriving at a response that is customized to the individual involves performing a series of complex analytical steps, such as: checking data availability, calculating average sleep duration, identifying sleep pattern anomalies over a period of time, contextualizing these findings within the individual's broader health, integrating knowledge of population norms of sleep, and offering tailored sleep improvement recommendations. Recently, we showed how building on Gemini models’ advanced capabilities in multimodality and long-context reasoning could enable state-of-the-art performance on a diverse set of medical tasks. However, such tasks rarely make use of complex data sourced from mobile and wearable devices relevant for personal health monitoring.
  
  Building on the next-generation capabilities of Gemini models, we present research that highlights two complementary approaches to providing accurate personal health and wellness information with LLMs. The first paper, “Towards a Personal Health Large Language Model”, demonstrates that LLMs fine-tuned on expert analysis and self-reported outcomes are able to successfully contextualize physiological data for personal health tasks. The second paper, “Transforming Wearable Data into Personal Health Insights Using Large Language Model Agents”, emphasizes the value of code generation and agent-based workflows to accurately analyze behavioral health data through natural language queries. We believe that bringing these ideas together, to enable interactive computation and grounded reasoning over personal health data, will be critical components for developing truly personalized health assistants. With these two papers, we curate new benchmark datasets across a range of personal health tasks, which help evaluate the effectiveness of these models.
  
  ###
  
  https://huggingface.co/collections/google/recurrentgemma-release-66152cbdd2d6619cb1665b7a
  📣 🧠 Exciting news for researchers pushing the boundaries of efficient deep learning! We've scaled RecurrentGemma to 9 billion parameters!
  🚀 This new model achieves performance comparable to the largest Gemma 1 model, but with significantly greater efficiency. That means lower memory requirements and faster sampling speeds, especially for long sequences or large batch sizes.
  For example, on a single TPU-v4, it delivers 80x higher throughput when sampling 1k tokens from a 2k token prompt.
  Model information
  Model summary
  Description
  RecurrentGemma is a family of open language models built on a novel recurrent architecture developed at Google. Both pre-trained and instruction-tuned versions are available in English.
  
  Like Gemma, RecurrentGemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Because of its novel architecture, RecurrentGemma requires less memory than Gemma and achieves faster inference when generating long sequences.
  
  ###
  
  https://www.microsoft.com/en-us/industry/blog/financial-services/insurance/2024/05/01/how-microsoft-copilot-for-microsoft-365-is-redefining-insurance-one-role-at-a-time/
  Insurance Thought leadership · 5 min read
  How Microsoft Copilot for Microsoft 365 is redefining insurance, one role at a time
  By Naveen Dhar, Director, Insurance Digital Strategy, Worldwide Financial Services
  May 1, 2024
  
  Financial services
  Azure
  more
  Insurers are facing greater challenges today than at any time in recent memory. Between economic and geo-political factors, climate change impacts, and the new social and competitive realities of a post-COVID-19 world, the insurance landscape is perilous—but also rich with opportunities. In response, innovative companies are exploring new business models and rethinking employee engagement, with technology at the core of new approaches.
  
  It is no wonder then that AI is so appealing to leading insurers. Its amazing ability to glean insights from data, create documents, and enable people to build powerful solutions using natural language promises to help insurers meet the unprecedented demands of the moment. This is central to our vision for intelligent insurance and our work with Microsoft Cloud for Financial Services.
  
  Most of the insurance companies we talk to are seriously exploring generative AI. For many, however, the scope of the opportunity is so wide-ranging that it can be difficult to identify productive early steps. Fortunately, there is an option available today that can deliver clear near-term productivity benefits and also help lay the groundwork for a successful long-term AI journey.
  
  How Microsoft Copilot for Microsoft 365 can impact insurance
  
  Imagine two years ago if a vendor promised a solution that resulted in employees spending 64% less time on email or that resulted in 70% of employees self-reporting as more productive. What seemed unbelievable until very recently is the impact of Microsoft Copilot for Microsoft 365 just months after its introduction last year.
  
  INTRODUCING COPILOT FOR MICROSOFT 365—A WHOLE NEW WAY TO WORK
  
  Read the blog
  Copilot for Microsoft 365 is a unique offering that integrates generative AI features into the Microsoft 365 applications that many employees use on a regular basis. In effect, it is a real-time, intelligent assistant built into Word, Excel, PowerPoint, Outlook, Microsoft Teams, and more, applying the power of large language models (LLMs) to an organization’s data to significantly reduce the time and energy required to perform an endless number of rote tasks.
  
  In November 2023, Microsoft surveyed 297 early adopters, across industries, to quantify the impact of Copilot for Microsoft 365. As detailed in the Work Trend Index Special Report, it delivered significant productivity gains among employees, with the promise of fostering broader organizational improvements over time. This is especially exciting for insurance companies, which have a wide range of roles that could benefit from this assistance.
  
  ###
  
  https://news.microsoft.com/ko-kr/2024/05/14/wtilaunch2024/?ocid=AID2445079_LINKEDIN_oo_spl100005641475500
  마이크로소프트, 링크드인과 Work Trend Index 2024 (업무동향지표) 발표
  2024년 5월 14일
  
  전체 근로자 4명 중 3명(75%) 직장에서 AI 활용 중, 6개월 전부터 AI를 사용한 비율도 46% 증가
  리더 79% AI 기술 적용 조직 경쟁력 유지 필수로 인식, 이들 중 71%는 경력보다 AI 역량 우선시
  AI 파워 유저 부상, 90% 이상 업무량 관리, 하루 일과(85%), AI 실험 의지(68%), 교육 기대(35%)
  M365 코파일럿 프롬프트 활용 신규 기능 공개, 링크드인, 50개 신규 AI 학습 7월 8일까지 제공
  마이크로소프트가 링크드인과 함께 제작한 연례 보고서 Work Trend Index 2024(업무동향지표)를 발표, AI를 통해 변화한 전 세계 업무 동향과 채용 방식에 대한 주요 인사이트를 공개했다.
  
  2024년에 접어들면서 업무에 AI를 활용하는 조직이 급격히 증가하고 있다. 이에 마이크로소프트는 ‘업무동향지표 2024’를 통해 AI가 업무와 노동 시장 전반에 미치는 영향에 대한 포괄적인 시각을 담은 세 가지 주요 트렌드를 공개했다. 한국을 포함한 전 세계 31개국 31,000명이 설문조사에 참여했으며, 마이크로소프트 365에서 수집된 수 조개의 생산성 신호, 링크드인의 노동 및 채용 트렌드, 포춘 500대 기업과의 협업을 통해 진행된 연구 결과가 반영됐다.
  
  사티아 나델라(Satya Nadella) 마이크로소프트 CEO 겸 이사회 의장은 “AI가 일자리 전반에 걸쳐 전문 지식을 누구나 쉽게 접하고 활용할 수 있도록 민주화하고 있다”며, “업무동향지표 2024 보고서는 모든 조직이 AI 기술을 활용해 더 나은 의사 결정과 협업을 가능하게 하며, 이를 통해 궁극적으로 비즈니스 성과를 개선할 수 있는 기회를 강조하고 있다”고 말했다.
  
  직장에서의 AI 수요 증가
  
  AI 시대가 열리면서 조직들은 기술 혁신의 중대한 도전에 직면했다. 특히 새로운 기술로 성장을 가속화하고 비용을 효율적으로 관리할 수 있는 세상에서 AI는 조직의 비즈니스 트랜스포메이션에도 큰 영향을 미친다.
  
  보고서에 따르면 전체 근로자 4명 중 3명이 직장에서 AI를 활용하고 있는 것으로 조사됐다. 근로자 75%(한국 73%)가 AI를 사용하고 있으며, 6개월 전부터 AI를 사용한 비율은 46% 증가했다. 리더의 79%가(한국 80%) AI 도입이 경쟁력 유지에 필수적이라고 인식하고 있으나, 이들 중 60%는(한국 68%) 조직 내 비전과 명확한 계획이 부족한 것에 대해 우려하고 있다고 답했다.
  
  또한 개인화된 AI 솔루션을 통해 업무 효율성을 높이고 있는 경우가 많은 것으로 나타났다. 이 같은 경향은 BYOAI(Bring Your Own AI)라고 불리며, 조직에서 사용자가 자신의 AI 도구를 개인적으로 사용하는 트렌드를 말한다. 실제로 근로자 78%(한국 85%)는 응답자는 회사의 지원 없이 AI를 개인적으로 업무에 활용하고 있다고 답했다.
  
  AI를 통한 커리어 장벽 극복
  
  AI의 도입이 빠르게 이뤄지면서 대부분의 기업들은 인재를 확보하는 데에 어려움을 겪고 있는 것으로 조사됐다. 특히 이 같은 문제는 사이버 보안, 엔지니어링, 크리에이티브 디자인 직무에서 심화되고 있다.
  
  먼저 채용 이유에 대한 우선순위가 달라졌다. 리더 중 과반수 이상(55%)이 인재 확보에 대해 우려하고 있으며, 66%(한국 70%)는 AI 기술을 보유하지 않은 지원자를 채용하지 않겠다고 답했다. 실제로 2023년 링크드인 프로필에 AI 관련 기술을 추가한 회원 수는 전년 대비 142배나 늘었으며, AI 관련 키워드가 언급된 공고의 지원자 수는 평균 17% 증가했다.
  
  리더 응답자의 71%는 경력 유무보다 AI 역량을 갖춘 지원자를 선호한다고 응답했다. 한국 리더들도 77%의 높은 비중으로 AI 역량을 채용 우선순위로 두고 있다.
  
  AI 파워 유저의 부상과 미래 시사점
  
  이번 연구에서는 AI 사용량이 적은 회의론자부터 사용 빈도가 높은 파워 유저, 초보자와 탐색자까지 네 가지 유형으로 표본 집단을 분류했다. 특히 파워 유저는 AI를 통해 업무 시간을 절약하는 등 비즈니스 프로세스와 방향을 재설정하고 있는 것으로 확인됐다. 이들 중 90% 이상이 AI 기술로 업무량을 더 수월하게 관리하고 있다. 특히 85%(한국 83%)는 AI로 하루를 시작하고 있으며, 다음날의 업무를 준비하는 데 AI를 활용하고 있다는 응답도 85%(한국 81%)에 달했다. CEO로부터 AI의 중요성에 대해 들은 경험이 있다는 응답은 61%(한국 42%)로 나타났다.
  
  동료와 프롬프트에 대해 더 자주 소통하는 비율도 평균 대비 40%(한국 23%) 높게 나타났다. AI 사용 방법을 자주 실험할 가능성이 68%(한국 68%) 더 높은 편이며, 직무에 특화된 AI 교육을 받을 가능성도 35%(한국 30%) 높은 것으로 조사됐다.
  
  보고서는 2024년이 AI가 직장에서 현실화되는 해가 될 것으로 내다보고 있다. 마이크로소프트는 보고서를 통해 조직이 단순한 업무 개선을 넘어, 비즈니스 모델 전반의 긍정적인 변화를 만들기 위해서는 AI를 활용해 성장을 가속화하고 비용을 관리하며, 고객에게 더 큰 가치를 제공할 수 있어야 한다고 제언했다.
  
  마이크로소프트는 리더들에게 주어진 기회가 직원들의 AI에 대한 관심을 실질적인 비즈니스 성과로 연결할 수 있다고 강조한다. 이를 위해 △조직 내 비즈니스 해결을 위한 AI 도입 △ 탑다운(Top-down), 바텀업(Bottom-up) 접근법을 통한 모든 직원의 커뮤니케이션 협력 강화 △맞춤형 교육 제공을 통한 AI 기술 습득 등 AI 시대 혁신을 위한 세 가지 주요 전략도 소개했다.
  
  조원우 한국마이크로소프트 대표는 “생성형 AI가 등장하면서 스마트폰 이후 처음으로 기술과 사람이 상호 작용하는 방식이 근본적으로 바뀌기 시작했다”며 “이번 보고서에서 눈여겨볼 만한 점은 AI가 적합한 인재 채용을 희망하는 리더와 경력 전환을 희망하는 직원에게 큰 기회를 제공할 것이라는 점”이라며, “AI 기술 도입에 따른 일자리 시장의 긍정적인 변화를 기대한다”고 전했다.
  
  이와 함께 마이크로소프트 365 코파일럿(Copilot for Microsoft 365) ‘자동 완성(Auto-complete)’ 신규 기능을 공개했다. 이를 통해 사용자는 본인이 입력한 프롬프트 텍스트를 기반으로 보다 유용한 제안을 받을 수 있게 됐다. 또한, 재작성(Rewriting) 기능은 회의, 문서 및 이메일을 기반으로 프롬프트를 보다 상세하게 변환해 준다. 새로운 채팅 인터페이스 캐치 업(Catch Up)은 사용자의 최근 업무를 기반으로 회의, 이메일 등 중요 문서와 기타 관련 정보를 표시한다.
  
  한편, 링크드인 러닝(LinkedIn Learning)을 통해 경력 관리를 위한 600개 이상의 AI 과정을 포함한 22,000개의 교육 과정을 제공하고 있다. 이번에 공개된 50여 개의 AI 강좌는 오는 7월 8일까지 무료로 수강할 수 있다. 또한, 새로운 AI 기반 코칭과 개인화된 직무 적합성 평가 도구를 통해 사용자가 빠르게 기술을 개발하고 적성에 맞는 직업을 찾을 수 있도록 돕고 있다.
  
  보다 자세한 내용은 마이크로소프트 공식 블로그와 업무동향지표 2024를 통해 확인 가능하다.
  
  ###
  
  https://huggingface.co/RLHFlow/ArmoRM-Llama3-8B-v0.1
  Home
  »
  Posts
  Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
  An interpretable reward modeling approach.
  May 29, 2024
  · 15 min · Haoxiang Wang |
  Suggest Changes
  Table of Contents
  Abstract
  Preliminaries
  RLHF Pipeline
  The Need for Interpretable Reward Models
  Multi-Objective Reward Modeling Meets Mixture-of-Experts
  Stage-1: Multi-Objective Reward Modeling
  Absolute-Rating Multi-Objective Reward Model (ArmoRM)
  Implementation of ArmoRM
  Stage-2: Mixture-of-Experts Aggregation of Reward Objectives
  ArmoRM with Mixture-of-Experts (Armo-MoE)
  Implementation of ArmoRM-MoE
  Empirical Results: SoTA on Reward-Bench
  Usage Example (Code Demo)
  Citation
  This work is authored by Haoxiang Wang*, Wei Xiong*, Tengyang Xie, Han Zhao, Tong Zhang (\* indicates equal contribution)
  
  Code: https://github.com/RLHFlow/RLHF-Reward-Modeling
  Model: https://huggingface.co/RLHFlow/ArmoRM-Llama3-8B-v0.1
  Technical Report: To be released in June, 2024
  Contact: Haoxiang Wang (hwang264@illinois.edu)
  Abstract
  Reinforcement learning from human feedback (RLHF) has emerged as the primary method for aligning large language models (LLMs) with human preferences. The RLHF process typically starts by training a reward model (RM) using human preference data. Conventional RMs are trained on pairwise responses to the same user request, with relative ratings indicating which response humans prefer. The trained RM serves as a proxy for human preferences. However, due to the black-box nature of RMs, their outputs lack interpretability, as humans cannot intuitively understand why an RM thinks a response is good or not. As RMs act as human preference proxies, we believe they should be human-interpretable to ensure that their internal decision processes are consistent with human preferences and to prevent reward hacking in LLM alignment. To build RMs with interpretable preferences, we propose a two-stage approach: i) train an Absolute-Rating Multi-Objective Reward Model (ArmoRM) with multi-dimensional absolute-rating data, each dimension corresponding to a human-interpretable objective (e.g., honesty, verbosity, safety); ii) employ a Mixture-of-Experts (MoE) strategy with a gating network that automatically selects the most suitable reward objectives based on the context. We efficiently trained an ArmoRM with Llama3-8B and a gating network consisting of a shallow MLP on top of the ArmoRM. Our final reward model, ArmoRM-Llama3-8B-v0.1, ranks first on the leaderboard of RewardBench, a benchmark evaluating RMs for language modeling. The performance of our model surpasses the LLM-as-a-judge approach using GPT-4 and the common Bradley-Terry modeling approach with Llama3-8B or Yi-34B by a margin.
  
  ###
  
  https://huggingface.co/papers/2406.06525
  Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
  Published on Jun 11
  ·
  Submitted by
  akhaliq
  on Jun 11
  #1 Paper of the day
  Authors:
  
  Peize Sun
  ,
  
  Yi Jiang
  ,
  
  Shoufa Chen
  ,
  
  Shilong Zhang
  ,
  Bingyue Peng
  ,
  Ping Luo
  ,
  
  Zehuan Yuan
  Abstract
  We introduce LlamaGen, a new family of image generation models that apply original ``next-token prediction'' paradigm of large language models to visual generation domain. It is an affirmative answer to whether vanilla autoregressive models, e.g., Llama, without inductive biases on visual signals can achieve state-of-the-art image generation performance if scaling properly. We reexamine design spaces of image tokenizers, scalability properties of image generation models, and their training data quality. The outcome of this exploration consists of: (1) An image tokenizer with downsample ratio of 16, reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet benchmark. (2) A series of class-conditional image generation models ranging from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256 benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A text-conditional image generation model with 775M parameters, from two-stage training on LAION-COCO and high aesthetics quality images, demonstrating competitive performance of visual quality and text alignment. (4) We verify the effectiveness of LLM serving frameworks in optimizing the inference speed of image generation models and achieve 326% - 414% speedup. We release all models and codes to facilitate open-source community of visual generation and multimodal foundation models.
  
  ###
  
  https://mhamilton.net/denseav
  
  Separating the "Chirp" from the "Chat":
  Self-supervised Visual Grounding
  of Sound and Language
  CVPR 2024
  Paper
  Code
  🤗 Demo
  Collab Notebook
  Dataset
  Mark Hamilton, Andrew Zisserman, John R. Hershey, William T. Freeman
  MIT, Microsoft, Google, University of oxford
  Abstract
  We present DenseAV, a novel dual encoder grounding architecture that learns high-resolution, semantically meaningful, and audio-visually aligned features solely through watching videos. We show that DenseAV can discover the `meaning'' of words and the `location'' of sounds without explicit localization supervision. Furthermore, it automatically discovers and distinguishes between these two types of associations without supervision. We show that DenseAV's localization abilities arise from a new multi-head feature aggregation operator that directly compares dense image and audio representations for contrastive learning. In contrast, many other systems that learn ``global'' audio and video representations cannot localize words and sound. Finally, we contribute two new datasets to improve the evaluation of AV representations through speech and sound prompted semantic segmentation. On these and other datasets we show DenseAV dramatically outperforms the prior art on speech and sound prompted semantic segmentation. DenseAV outperforms the previous state-of-the-art, ImageBind, on cross-modal retrieval using fewer than half of the parameters.