克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

简介

PaddleMIX是基于飞桨的跨模态大模型开发套件,聚合图像、文本、视频等多种模态,覆盖视觉语言预训练,文生图,文生视频等丰富的跨模态任务。提供开箱即用的开发体验,同时满足开发者灵活定制需求,探索通用人工智能。

最新进展

2023.10.7 发布 PaddleMIX v1.0

  • 新增图文预训练模型分布式训练能力,BLIP-2支持千亿规模训练
  • 新增跨模态应用流水线AppFlow,一键支持自动标注,图像编辑,音生图等11种跨模态应用
  • PPDiffusers发布 0.19.3 版本,新增SDXL及相关任务

2023.7.31 发布 PaddleMIX v0.1

  • 首次发布PaddleMIX跨模态大模型开发套件,融合PPDiffusers多模态扩散模型工具箱,广泛支持PaddleNLP大语言模型
  • 新增EVA-CLIP,BLIP-2,miniGPT-4,Stable Diffusion,ControlNet等12个跨模态大模型

主要特性

  • 丰富的多模态功能: 覆盖图文预训练,文生图,跨模态视觉任务,实现图像编辑、图像描述、数据标注等多样功能
  • 简洁的开发体验: 模型统一开发接口,高效实现自定义模型开发和功能实现
  • 高效的训推流程: 全量模型打通训练推理一站式开发流程,BLIP-2,Stable Diffusion等重点模型训推性能业界领先
  • 超大规模训练支持: 可训练千亿规模图文预训练模型,百亿规模文生图底座模型

任务展示

  • 视频Demo展示(video Demo)

https://github.com/PaddlePaddle/PaddleMIX/assets/29787866/8d32722a-e307-46cb-a8c0-be8acd93d2c8

安装

  1. 环境依赖
pip install -r requirements.txt

关于PaddlePaddle安装的详细教程请查看Installation

注:ppdiffusers部分模型需要依赖 CUDA 11.2 及以上版本,如果本地机器不符合要求,建议前往 AI Studio 进行模型训练、推理任务。

如果希望使用bf16训练推理,请使用支持bf16的GPU,如A100。

  1. 手动安装
git clone https://github.com/PaddlePaddle/PaddleMIX
cd PaddleMIX
pip install -e .

#ppdiffusers 安装
cd ppdiffusers
pip install -e .

教程

特色应用

  1. 艺术风格二维码模型

体验专区: https://aistudio.baidu.com/community/app/1339

  1. Mix叠图

体验专区: https://aistudio.baidu.com/community/app/1340

模型库

多模态预训练 扩散类模型
  • 图文预训练
  • 开放世界视觉模型
  • 更多模态预训练模型
  • 文生图
  • 文生视频
  • 音频生成
  • 更多模型能力,可参考模型能力矩阵

    许可证书

    本项目的发布受Apache 2.0 license许可认证。

    # Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from __future__ import absolute_import from __future__ import print_function from __future__ import unicode_literals import argparse import io import re import sys import os import datetime COPYRIGHT = '''Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.''' def _generate_copyright(comment_mark): copyright=COPYRIGHT.split(os.linesep) header = copyright[0].rstrip() p = re.search('(\d{4})', header).group(0) now = datetime.datetime.now() header = header.replace(p,str(now.year)) ans=[comment_mark + " " + header + os.linesep] for idx, line in enumerate(copyright[1:]): ans.append(comment_mark + " " + line.rstrip() + os.linesep) return ans def _get_comment_mark(path): lang_type=re.compile(r"\.(py|sh)$") if lang_type.search(path) is not None: return "#" lang_type=re.compile(r"\.(h|c|hpp|cc|cpp|cu|go|cuh|proto)$") if lang_type.search(path) is not None: return "//" return None RE_ENCODE = re.compile(r"^[ \t\v]*#.*?coding[:=]", re.IGNORECASE) RE_COPYRIGHT = re.compile(r".*Copyright( \(c\))* \d{4}", re.IGNORECASE) RE_SHEBANG = re.compile(r"^[ \t\v]*#[ \t]?\!") def _check_copyright(path): head=[] try: with open(path) as f: head = [next(f) for x in range(4)] except StopIteration: pass for idx, line in enumerate(head): if RE_COPYRIGHT.search(line) is not None: return True return False def generate_copyright(path, comment_mark): original_contents = io.open(path, encoding="utf-8").readlines() head = original_contents[0:4] insert_line_no=0 for i, line in enumerate(head): if RE_ENCODE.search(line) or RE_SHEBANG.search(line): insert_line_no=i+1 copyright = _generate_copyright(comment_mark) if insert_line_no == 0: new_contents = copyright if len(original_contents) > 0 and len(original_contents[0].strip()) != 0: new_contents.append(os.linesep) new_contents.extend(original_contents) else: new_contents=original_contents[0:insert_line_no] new_contents.append(os.linesep) new_contents.extend(copyright) if len(original_contents) > insert_line_no and len(original_contents[insert_line_no].strip()) != 0: new_contents.append(os.linesep) new_contents.extend(original_contents[insert_line_no:]) new_contents="".join(new_contents) with io.open(path, 'w') as output_file: output_file.write(new_contents) def main(argv=None): parser = argparse.ArgumentParser( description='Checker for copyright declaration.') parser.add_argument('filenames', nargs='*', help='Filenames to check') args = parser.parse_args(argv) retv = 0 for path in args.filenames: comment_mark = _get_comment_mark(path) if comment_mark is None: print("warning:Unsupported file", path, file=sys.stderr) continue if _check_copyright(path): continue generate_copyright(path, comment_mark) if __name__ == '__main__': exit(main())

    简介

    方便国内环境下载 展开 收起
    Python 等 5 种语言
    Apache-2.0
    取消

    发行版

    暂无发行版

    贡献者

    全部

    近期动态

    不能加载更多了
    马建仓 AI 助手
    尝试更多
    代码解读
    代码找茬
    代码优化