기본 정보
연구 분야
프로젝트
논문
구성원
article|
gold
·인용수 0
·2024
From Generation to Selection: Findings of Converting Analogical Problem-Solving into Multiple-Choice Questions
Donghyeon Shin, Seungpil Lee, Klea Lena Kovacec, Sundong Kim
초록

As artificial intelligence reasoning abilities gain prominence, generating reliable benchmarks becomes crucial.The Abstract and Reasoning Corpus (ARC) offers challenging problems yet unsolved by AI.While ARC effectively assesses reasoning, its generation-based evaluation overlooks other assessment aspects.Bloom's Taxonomy suggests evaluating six cognitive stages: Remember, Understand, Apply, Analyze, Evaluate, and Create.To extend ARC's focus beyond the Create stage, we developed MC-LARC, a multiple-choice format suitable for assessing stages like Understand and Apply in Large Language Models (LLMs).Our evaluation of ChatGPT4V's analogical reasoning using MC-LARC confirmed that this format supports LLMs' reasoning capabilities and facilitates evidence analysis.However, we observed LLMs using shortcuts in MC-LARC tasks.To address this, we propose a self-feedback framework where LLMs identify issues and generate improved options.

키워드
Selection (genetic algorithm)Computer scienceAnalogical reasoningArtificial intelligenceMathematical optimizationMathematicsAnalogyEpistemologyPhilosophy
타입
article
IF / 인용수
- / 0
게재 연도
2024

주식회사 디써클

대표 장재우,이윤구서울특별시 강남구 역삼로 169, 명우빌딩 2층 (TIPS타운 S2)대표 전화 0507-1312-6417이메일 info@rndcircle.io사업자등록번호 458-87-03380호스팅제공자 구글 클라우드 플랫폼(GCP)

© 2026 RnDcircle. All Rights Reserved.