dc.contributor.author | Hazman, Muzhaffar | |
dc.contributor.author | McKeever, Susan | |
dc.contributor.author | Griffith, Josephine | |
dc.contributor.editor | Longo, L | |
dc.contributor.editor | O'Reilly, R | |
dc.date.accessioned | 2023-01-20T11:18:07Z | |
dc.date.available | 2023-01-20T11:18:07Z | |
dc.date.issued | 2022-12-08 | |
dc.identifier.citation | Hazman, Muzhaffar, McKeever, Susan, & Griffith, Josephine. (2022). Meme sentiment analysis enhanced with multimodal spatial encoding and facial embedding. Paper presented at the 30th Irish Conference on Artificial Intelligence and Cognitive Science, Cork, Ireland, 08-09 December, https://doi.org/10.13025/6d0s-tc11 | en_IE |
dc.identifier.uri | http://hdl.handle.net/10379/17643 | |
dc.description.abstract | Internet memes are characterised by the interspersing of text amongst visual elements. State-of-the-art multimodal meme classifiers do not account for the relative positions of these elements across the two modalities, despite the latent meaning associated with where text and visual elements are placed. Against two meme sentiment classification datasets, we systematically show performance gains from incorporating the spatial position of visual objects, faces, and text clusters extracted from memes. In addition, we also present facial embedding as an impactful enhancement to image representation in a multimodal meme classifier. Finally, we show that incorporating this spatial information allows our fully automated approaches to outperform their corresponding baselines that rely on additional human validation of OCR-extracted text. | en_IE |
dc.description.sponsorship | This work was conducted with the financial support of the Science Foundation
Ireland Centre for Research Training in Digitally-Enhanced Reality (d-real) under
Grant No. 18/CRT/6224. | en_IE |
dc.format | application/pdf | en_IE |
dc.language.iso | en | en_IE |
dc.publisher | University of Galway | en_IE |
dc.relation.ispartof | 30th Irish Conference on Artificial Intelligence and Cognitive Science | en |
dc.rights | CC BY-NC-ND 3.0 IE | |
dc.rights.uri | https://creativecommons.org/licenses/by-nc-nd/3.0/ie/ | |
dc.subject | Internet Meme | en_IE |
dc.subject | Multimodal Classification | en_IE |
dc.subject | Deep Learning | en_IE |
dc.subject | Positional Embedding | en_IE |
dc.subject | Sentiment Analysis | en_IE |
dc.title | Meme sentiment analysis enhanced with multimodal spatial encoding and facial embedding | en_IE |
dc.type | Conference Paper | en_IE |
dc.date.updated | 2023-01-20T10:26:15Z | |
dc.identifier.doi | 10.13025/6d0s-tc11 | |
dc.local.publishedsource | https://doi.org/10.13025/6d0s-tc11 | |
dc.description.peer-reviewed | peer-reviewed | |
dc.contributor.funder | Science Foundation Ireland | en_IE |
dc.internal.rssid | 29991239 | |
dc.local.contact | Muzhaffar Hazman, It308, 3rd Floor, It Building, Nui Galway. Email: m.sukiman1@nuigalway.ie | |
dc.local.copyrightchecked | Yes | |
dc.local.version | ACCEPTED | |
nui.item.downloads | 45 | |