Paper Notes - Active Learning
Active Learning 也称为查询学习或者最优实验设计. 主动学习通过设计合理的查询函数, 不断从未标注的数据中挑选出数据标注后放入训练集. 有效的主动学习数据选择策...
Paper Notes
Paper Notes - Attention
1. Attention 机制 1a. 背景知识 我们最为熟悉的NMT模型便是经典的Seq2Seq, 这篇文章从一个Seq2Seq模型开始介绍, 然后进一步看如何将Attent...
Paper Notes - Multi-Teacher Knowledge Distillation - 1
Learning from Multiple Teacher Networks http://library.usc.edu.ph/ACM/KKD%202017/pdfs/p1285.pdf loss: teachers的softmax输出取平均和student的交叉熵 中间层表示的相对相异度(仅适用于MTKD), 三元组$(q_i...
Paper Notes - Multi-Teacher Knowledge Distillation - 2
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks https://arxiv.org/pdf/2004.05937.pdf Learning from Multiple Teacher Networks, KDD 2017 Efficient knowledge distillation from an ensemble of teachers. Interspeech 2017: 对teacher的logits取加权平均, 加权平均和student的logit...
Paper Notes - Object Detection
Definition Image Classification: 输入图片, 输出图中目标物体的类别. Object Localization: 输入图片, 输出图中物体的 bounding box. Object Detection: 输入图片, 输出图中物体的 bounding box 和类别. R-CNN Model Family 采用region proposal methods, 首...