1 Ocak 2023 Dergi makalesi Açık Erişim

Keser, Reyhan Kevser; Ayanzadeh, Aydin; Aghdam, Omid Abdollahi; Kilcioglu, Caglar; Toreyin, Behcet Ugur; Ure, Nazim Kemal

MARC21 XML

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nam##2200000uu#4500</leader>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">user-tubitak-destekli-proje-yayinlari</subfield>
    <subfield code="o">oai:aperta.ulakbim.gov.tr:253867</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">One of the most efficient methods for model compression is hint distillation, where the student model is injected with information (hints) from several different layers of the teacher model. Although the selection of hint points can drastically alter the compression performance, conventional distillation approaches overlook this fact and use the same hint points as in the early studies. Therefore, we propose a clustering based hint selection methodology, where the layers of teacher model are clustered with respect to several metrics and the cluster centers are used as the hint points. Our method is applicable for any student network, once it is applied on a chosen teacher network. The proposed approach is validated in CIFAR-100 and ImageNet datasets, using various teacher-student pairs and numerous hint distillation methods. Our results show that hint points selected by our algorithm results in superior compression performance compared to state-of-the-art knowledge distillation algorithms on the same student models and datasets.</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">article</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="a">Creative Commons Attribution</subfield>
    <subfield code="u">http://www.opendefinition.org/licenses/cc-by</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="a">Keser, Reyhan Kevser</subfield>
    <subfield code="u">Istanbul Tech Univ, Informat Inst, Signal Proc Computat Intelligence Res Grp SP4CING, Istanbul, Turkiye</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="z">md5:cb26c3956930ca10c358892d25da70df</subfield>
    <subfield code="s">229</subfield>
    <subfield code="u">https://aperta.ulakbim.gov.trrecord/253867/files/bib-0effc38a-bd14-4576-b718-9a2a86827653.txt</subfield>
  </datafield>
  <controlfield tag="005">20230728195424.0</controlfield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2023-01-01</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.1016/j.eswa.2022.119040</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="4">
    <subfield code="v">213</subfield>
    <subfield code="p">EXPERT SYSTEMS WITH APPLICATIONS</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Ayanzadeh, Aydin</subfield>
    <subfield code="u">Univ Maryland, Dept Comp Sci &amp; Elect Engn, Baltimore, MD USA</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Aghdam, Omid Abdollahi</subfield>
    <subfield code="u">Arcelik Res &amp; Dev, Istanbul, Turkiye</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Kilcioglu, Caglar</subfield>
    <subfield code="u">Arcelik Res &amp; Dev, Istanbul, Turkiye</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Toreyin, Behcet Ugur</subfield>
    <subfield code="u">Istanbul Tech Univ, Informat Inst, Signal Proc Computat Intelligence Res Grp SP4CING, Istanbul, Turkiye</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Ure, Nazim Kemal</subfield>
    <subfield code="u">Istanbul Tech Univ, Artificial Intelligence &amp; Data Sci Applicat &amp; Res, Istanbul, Turkiye</subfield>
  </datafield>
  <controlfield tag="001">253867</controlfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-tubitak-destekli-proje-yayinlari</subfield>
  </datafield>
</record>