本文复现DETR目标检测方案，基于Transformer实现端到端检测，无需NMS或anchor生成。模型含CNN（Res50）+Transformer+FFN结构，用二分图匹配与匈牙利算法处理预测框。适配Paddle 2.0，修复BUG并对齐精度，DETR-DC5变体在COCO2017验证集MAP达0.431。

模型介绍

论文复现第三期DETR方案， 精确度MAP为0.431

End-to-End Object Detection with Transformers

参考代码: https://aistudio.baidu.com/aistudio/projectdetail/1327221

• 在大佬的基础上进行修改，包括Paddle 2.0适配， BUG修复，以及精度对齐等工作

github pytorch代码:?https://github.com/facebookresearch/detr

论文地址:?https://arxiv.org/pdf/1706.03762.pdf

• 基于transformer实现端到端目标检测, 两大主要组成，一者是预测的匹配损失函数，再者预测目标和他们的关系。
• 不需要使用NMS或者anchor生成， 本文能将该任务的先验知识进行编码。DETR结合目标关联关系和上下文直接并行输出最终的预测集。
• Bipartite Matching二分图最大匹配， 将节点集V分割为互不相割的子集。训练时将预测框和GT boxes进行匹配。若没有找到匹配的预测框作为"no object"。 最终计算匹配损失。
• transformer的自注意力机制，专门对序列中成对交互的元素进行建模，所以也很好解决去除重复框
• 推理出的box坐标是直接基于输入图片而不是基于anchor
• 先预设固定数量的预测量，通常比图像的目标数量多。南调是对预测目标进行评分，使用最低成对匹配成本，用到匈牙利算法。线性组合IOU损失和L1损失。
• 网络结构由cnn(res50)+transformer+FFN. 常用的backbone都可以用。cnn到transformer通过1x1卷积进行降维，空间维度压缩到一维， FFN由3层MLP+relu组成。
• backbone和transformer分开训练，学习率设置不一样。具体是怎样的方式？
• 关于transformer，
  • 其实就是全连接(或一维卷积)加上attention结合体，对于局部特征捕获能力稍欠缺。
  • 编解码结构较大的局限性是固定长度的语义向量， 编码器压缩成一个固长向量无法表示整个序列信息, 前期信息容易被后序信息覆盖，解码时输入每个单词权重不一致。
  • attention解决编解码结构的弊端，编码器时将正序和逆序的隐藏层状态结合起来， 解码时步骤，1.计算编码器隐藏层状态和解码器隐藏层状态间的相关程度并用softmax归一化得到权重值， 2. 计算语义编码向量的加权和，3. 根据加权和计算隐藏状态和解码器输出。不同编解码结构的是解码时选择性的从向量序列中挑选。
  • soft attention缺点是每次decode都会计算所有编码器隐藏层状态向量，计算复杂度较高。hard attention每次近选择一个source进行计算，缺点是不可微，无法反向传播。
  • global attetion也属于soft attention， 只是计算权重公式上有差别。实验表明general方式好一些。
  • local attention, 每次选择一部分source进行计算， 既减少计算量又能可微。思路是为decoder预测一个source位置，然后基于位置选择一个窗口用于计算编码向量。关键是怎么确定pt位置， monotoic或predictive。
  • self attention, 传统attention是基于target和source间的相似度，而self是发生在target内部或者source内部的相似关联。self更容易捕获长距依赖特征。query和key点积相当于给每个单词打分，决定了编码单词时重视句子其他部分的程度。softmax决定了其他单词对当前单词编码的贡献，self作为最大，其他单词也很有参考作用。
  • transformer和attention机制一样，只是更复杂，多个encoder和docoder堆叠一起， encoder包含self-attention(不仅仅当前词还有上下文)和神经网络层(可并行)，decoder多了一层attention(当前需关注的重点内容)。encoder前要对输入数据进行embedding操作。
  • 位置编码， transformer缺少解释序列中单词顺序的方法，positional encoding在维度上和embedding一样，偶数位置正弦编码，奇数位置余弦编码，决定当前词的位置或不同词间距离，并且模型可学习到。
  • 多头注意力，主要扩展了模型专注不同位置的能力，给出了注意力层的多个表示子空间。8个头就有8套qkv矩阵，每个头的qkv矩阵权重独立，将这些矩阵拼接在一起并用附加权重矩阵相乘。
  • transformer还使用到了残差和layer norm。LN是在每一个样本上计算均值和方差，而不是BN那种在批方向计算均值和方差。还用到了mask，掩盖值，使其参数更新时不产生效果， padding mask在较短序列填充0，Sequence mask只依赖t之前的信息，将t之后信息掩盖起来。

关于数据集COCO2017

COCO的全称是Common Objects in Context，是微软团队提供的一个可以用来进行图像识别的数据集。MS COCO数据集中的图像分为训练、验证和测试集。其行业地位就不再多少了，本文主要梳理一下该数据集包含的内容。下图是官网给出的可下载的数据集（更新时间2020年01月09日），从这里可看出其数据集主要包括有标注的和无标注的数据。

#解压数据集%cd /home/aistudio/data/data7122/ !unzip train2017.zip!unzip val2017.zip !unzip annotations_trainval2017.zip登录后复制 ? ?In [?]

#加载数据集%cd ~/my_detr!python coco_dataset.py登录后复制 ? ?

模型结构搭建

1. Backbone在resnet50基础上修改，支持DC5变体
2. 搭建transformer结构
3. 搭建后处理包括匈牙利匹配算法
4. 损失函数
5. 后处理

核心代码主要有:

• model.py
• resnet.py
• backbone.py
• transformer.py

<br/>登录后复制 ? ?

精度对齐

因为loss函数以及后处理较为复杂，所以这里特定还用了loss精度对齐

输出结果结果精度对齐

模拟tensor输入: image = [paddle.ones(shape=[3, 800, 1199])] samples.tensors = torch.ones(1, 3, 800, 1199) 因为resize对图像输入有一定差距，所以使用ones的张量, 比如使用样例数据, 0.348(torch) vs 0.31(paddle)

pytorch结果:

paddle结果:

LOSS 精度对齐

相差小数点2位

Pytorch的结果:

PaddlePaddle的结果:

训练DETR

#开始训练%cd ~/my_detr!python train_val.py train登录后复制 ? ? ? ?

/home/aistudio/my_detr/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations def convert_to_list(value, n, name, dtype=np.int):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Sized---->{'hidden_dim': 256, 'lr_backbone': -1, 'masks': False, 'dilation': False, 'backbone': 'resnet50', 'num_classes': 91, 'dropout': 0.1, 'nheads': 8, 'dim_feedforward': 2048, 'enc_layers': 6, 'dec_layers': 6, 'pre_norm': False, 'num_queries': 100, 'aux_loss': True, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.1, 'coco_path': '/home/aistudio/data/data7122', 'lr': 1e-06, 'clip_max_norm': 0.1, 'batch_size': 8, 'epochs': 2}{'num_classes': 91, 'norm_layer': <class 'backbone.FrozenBatchNorm2d'>}block <class 'resnet.BottleneckBlock'>[False, False, False]W0514 19:16:37.620630 26619 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1W0514 19:16:37.627422 26619 device_context.cc:372] device: 0, cuDNN Version: 7.6.debug: 1 1debug: 2 1debug: 2 1debug: 2 1Epoch 0: StepDecay set learning rate to 1e-06.loading annotations into memory...Done (t=0.56s)creating index...index created!loading annotations into memory...Done (t=0.69s)creating index...index created!/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:687: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations elif dtype == np.bool:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:143: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations if data.dtype == np.object:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:238: UserWarning: The dtype of left and right variables are not the same, left dtype is VarType.FP32, but right dtype is VarType.INT64, the right dtype will convert to VarType.FP32 format(lhs_dtype, rhs_dtype, lhs_dtype))epoch: 0, batch_id: 0, loss: 10.05978012084961epoch: 0, batch_id: 100, loss: 6.360683917999268epoch: 0, batch_id: 200, loss: 6.134881973266602epoch: 0, batch_id: 300, loss: 7.114040851593018epoch: 0, batch_id: 400, loss: 6.500746250152588epoch: 0, batch_id: 500, loss: 6.7673797607421875epoch: 0, batch_id: 600, loss: 5.745387554168701epoch: 1, batch_id: 0, loss: 7.610352993011475epoch: 1, batch_id: 100, loss: 5.726753234863281epoch: 1, batch_id: 200, loss: 5.837918758392334epoch: 1, batch_id: 300, loss: 6.997137069702148epoch: 1, batch_id: 400, loss: 6.292409420013428epoch: 1, batch_id: 500, loss: 6.677578926086426epoch: 1, batch_id: 600, loss: 5.692938327789307登录后复制 ? ? ? ?

验证DETR

%cd ~/my_detr!python train_val.py eval登录后复制 ? ? ? ?

/home/aistudio/my_detr/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations def convert_to_list(value, n, name, dtype=np.int):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Sized---->{'hidden_dim': 256, 'lr_backbone': -1, 'masks': False, 'dilation': False, 'backbone': 'resnet50', 'num_classes': 91, 'dropout': 0.1, 'nheads': 8, 'dim_feedforward': 2048, 'enc_layers': 6, 'dec_layers': 6, 'pre_norm': False, 'num_queries': 100, 'aux_loss': True, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.1, 'coco_path': '/home/aistudio/data/data7122', 'lr': 1e-06, 'clip_max_norm': 0.1, 'batch_size': 8, 'epochs': 2}{'num_classes': 91, 'norm_layer': <class 'backbone.FrozenBatchNorm2d'>}block <class 'resnet.BottleneckBlock'>[False, False, False]W0514 21:08:15.278102 7932 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1W0514 21:08:15.283421 7932 device_context.cc:372] device: 0, cuDNN Version: 7.6.debug: 1 1debug: 2 1debug: 2 1debug: 2 1Epoch 0: StepDecay set learning rate to 1e-06.loading annotations into memory...Done (t=0.55s)creating index...index created!loading annotations into memory...Done (t=0.68s)creating index...index created!/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:687: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations elif dtype == np.bool:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:238: UserWarning: The dtype of left and right variables are not the same, left dtype is VarType.FP32, but right dtype is VarType.INT64, the right dtype will convert to VarType.FP32 format(lhs_dtype, rhs_dtype, lhs_dtype))Accumulating evaluation results.../opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:378: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations tp_sum = np.cumsum(tps, axis=1).astype(dtype=np.float)/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:379: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations fp_sum = np.cumsum(fps, axis=1).astype(dtype=np.float)DONE (t=11.61s).IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.420 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.624 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.442 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.213 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.460 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.334 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.533 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.575 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.325 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.631 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.809登录后复制 ? ? ? ?

训练DETR-DC5

这个模型非常耗显存, 很难训练

#开始训练DC5%cd ~/my_detr!python train_val_dc5.py train登录后复制 ? ? ? ?

/home/aistudio/my_detr/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations def convert_to_list(value, n, name, dtype=np.int):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Sized---->{'hidden_dim': 256, 'lr_backbone': -1, 'masks': False, 'dilation': True, 'backbone': 'resnet50', 'num_classes': 91, 'dropout': 0.1, 'nheads': 8, 'dim_feedforward': 2048, 'enc_layers': 6, 'dec_layers': 6, 'pre_norm': False, 'num_queries': 100, 'aux_loss': True, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.1, 'coco_path': '/home/aistudio/data/data7122', 'lr': 1e-06, 'clip_max_norm': 0.1, 'batch_size': 1, 'epochs': 1}{'num_classes': 91, 'replace_stride_with_dilation': [False, False, True], 'norm_layer': <class 'backbone.FrozenBatchNorm2d'>}block <class 'resnet.BottleneckBlock'>[False, False, True]W0514 17:11:00.507395 24269 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1W0514 17:11:00.512784 24269 device_context.cc:372] device: 0, cuDNN Version: 7.6.debug: 1 1debug: 2 1debug: 2 1debug: 1 2530Epoch 0: StepDecay set learning rate to 1e-06.loading annotations into memory...Done (t=0.57s)creating index...index created!loading annotations into memory...Done (t=0.70s)creating index...index created!/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:687: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations elif dtype == np.bool:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:143: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations if data.dtype == np.object:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:238: UserWarning: The dtype of left and right variables are not the same, left dtype is VarType.FP32, but right dtype is VarType.INT64, the right dtype will convert to VarType.FP32 format(lhs_dtype, rhs_dtype, lhs_dtype))epoch: 0, batch_id: 0, loss: 5.015964031219482epoch: 0, batch_id: 100, loss: 14.247769355773926epoch: 0, batch_id: 200, loss: 7.298154830932617epoch: 0, batch_id: 300, loss: 1.0471426248550415epoch: 0, batch_id: 400, loss: 7.584997653961182epoch: 0, batch_id: 500, loss: 3.577380895614624epoch: 0, batch_id: 600, loss: 4.61794900894165epoch: 0, batch_id: 700, loss: 5.049403667449951epoch: 0, batch_id: 800, loss: 5.122508525848389epoch: 0, batch_id: 900, loss: 3.216052770614624epoch: 0, batch_id: 1000, loss: 3.3042514324188232epoch: 0, batch_id: 1100, loss: 4.413068771362305epoch: 0, batch_id: 1200, loss: 7.288424015045166epoch: 0, batch_id: 1300, loss: 6.1409735679626465epoch: 0, batch_id: 1400, loss: 10.504143714904785epoch: 0, batch_id: 1500, loss: 3.685210704803467epoch: 0, batch_id: 1600, loss: 2.6168665885925293epoch: 0, batch_id: 1700, loss: 22.14373016357422epoch: 0, batch_id: 1800, loss: 8.267280578613281epoch: 0, batch_id: 1900, loss: 1.4486600160598755epoch: 0, batch_id: 2000, loss: 4.107017993927002epoch: 0, batch_id: 2100, loss: 9.582965850830078epoch: 0, batch_id: 2200, loss: 6.967478275299072epoch: 0, batch_id: 2300, loss: 15.338693618774414epoch: 0, batch_id: 2400, loss: 13.399685859680176epoch: 0, batch_id: 2500, loss: 1.4326478242874146epoch: 0, batch_id: 2600, loss: 6.990074157714844epoch: 0, batch_id: 2700, loss: 8.32422161102295epoch: 0, batch_id: 2800, loss: 5.453993797302246epoch: 0, batch_id: 2900, loss: 7.272365093231201epoch: 0, batch_id: 3000, loss: 8.217702865600586epoch: 0, batch_id: 3100, loss: 2.2091081142425537epoch: 0, batch_id: 3200, loss: 0.9168111085891724epoch: 0, batch_id: 3300, loss: 1.9928405284881592epoch: 0, batch_id: 3400, loss: 6.245678424835205epoch: 0, batch_id: 3500, loss: 8.91486930847168epoch: 0, batch_id: 3600, loss: 6.916267395019531epoch: 0, batch_id: 3700, loss: 4.461080551147461epoch: 0, batch_id: 3800, loss: 8.265044212341309epoch: 0, batch_id: 3900, loss: 8.603659629821777epoch: 0, batch_id: 4000, loss: 4.085428714752197epoch: 0, batch_id: 4100, loss: 10.250466346740723epoch: 0, batch_id: 4200, loss: 2.327882766723633epoch: 0, batch_id: 4300, loss: 3.273315906524658epoch: 0, batch_id: 4400, loss: 6.719542026519775epoch: 0, batch_id: 4500, loss: 3.232994794845581epoch: 0, batch_id: 4600, loss: 4.106349945068359epoch: 0, batch_id: 4700, loss: 1.6361501216888428epoch: 0, batch_id: 4800, loss: 4.615266799926758epoch: 0, batch_id: 4900, loss: 8.394232749938965start evaluating....Accumulating evaluation results.../opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:378: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations tp_sum = np.cumsum(tps, axis=1).astype(dtype=np.float)/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:379: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations fp_sum = np.cumsum(fps, axis=1).astype(dtype=np.float)DONE (t=9.37s).IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.432 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.634 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.458 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.223 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.471 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.612 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.340 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.551 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.591 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.334 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.642 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.815save weights with map: 0.43226377363430357登录后复制 ? ? ? ?

验证DETR-DC5

#验证DETR-DC5%cd ~/my_detr!python train_val_dc5.py eval登录后复制 ? ? ? ?

/home/aistudio/my_detr/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations def convert_to_list(value, n, name, dtype=np.int):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Sized---->{'hidden_dim': 256, 'lr_backbone': -1, 'masks': False, 'dilation': True, 'backbone': 'resnet50', 'num_classes': 91, 'dropout': 0.1, 'nheads': 8, 'dim_feedforward': 2048, 'enc_layers': 6, 'dec_layers': 6, 'pre_norm': False, 'num_queries': 100, 'aux_loss': True, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.1, 'coco_path': '/home/aistudio/data/data7122', 'lr': 1e-06, 'clip_max_norm': 0.1, 'batch_size': 1, 'epochs': 1}{'num_classes': 91, 'replace_stride_with_dilation': [False, False, True], 'norm_layer': <class 'backbone.FrozenBatchNorm2d'>}block <class 'resnet.BottleneckBlock'>[False, False, True]W0514 21:21:04.015319 9341 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1W0514 21:21:04.020784 9341 device_context.cc:372] device: 0, cuDNN Version: 7.6.debug: 1 1debug: 2 1debug: 2 1debug: 1 2Epoch 0: StepDecay set learning rate to 1e-06.loading annotations into memory...Done (t=0.56s)creating index...index created!loading annotations into memory...Done (t=0.69s)creating index...index created!/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:687: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations elif dtype == np.bool:/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:238: UserWarning: The dtype of left and right variables are not the same, left dtype is VarType.FP32, but right dtype is VarType.INT64, the right dtype will convert to VarType.FP32 format(lhs_dtype, rhs_dtype, lhs_dtype))Accumulating evaluation results.../opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:378: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations tp_sum = np.cumsum(tps, axis=1).astype(dtype=np.float)/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/pycocotools/cocoeval.py:379: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations fp_sum = np.cumsum(fps, axis=1).astype(dtype=np.float)DONE (t=9.90s).IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.431 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.630 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.458 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.223 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.470 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.609 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.340 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.549 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.593 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.339 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.645 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.812登录后复制 ? ? ? ?