vibe-openscad

3D 空間認識ベンチマーク

複数の AI モデル / ハーネスに同じ課題で OpenSCAD コードを生成させ、結果を横並びで比較する。

課題
9 / 9
run
1084 (1031 ok / 53 fail)
累計コスト
$20.37 内 $0.3130 は失敗
累計時間
573.0m 内 51.1m は失敗

中央に貫通穴を持つ立方体 tier 1 tier-1-cube-with-hole

bare
anthropic
bare render of tier-1-cube-with-hole
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 39.1s 3739/1326t $0.0518
openai
google
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

階段状ピラミッド tier 1 tier-1-stepped-pyramid

bare
anthropic
bare render of tier-1-stepped-pyramid
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 33.8s 5071/1942t $0.0739
openai
google
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

取手付きマグカップ tier 1 tier-1-mug +6 過去 run

bare
anthropic
bare render of tier-1-mug
bare
claude opus 4.7
effort
effort=low low
medium
· high* render effort=max max · xhigh render
thinking
· off render · adaptive* render
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 90.0s 6128/6453t $0.1920
bare render of tier-1-mug
bare
claude sonnet 4.6
effort
effort=low low
medium
· high* render
max
xhigh
thinking
· off render thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
9 step 109.3s 6387/6418t $0.1154
openai
bare render of tier-1-mug
bare
openai gpt 5.4 nano 2026-03-17
effort
effort=low low effort=medium medium* · high render
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 112.2s 5210/14846t $0.0266
bare render of tier-1-mug
bare
openai gpt 5 mini 2025-08-07
effort
effort=low low effort=medium medium* effort=high high
max
xhigh
iterpng
iter 1 i1 · i2 render iter 3 i3
7 step 852.9s 4301/37053t $0.0752
google
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

六角ボルト M8x30 tier 2 tier-2-hex-bolt

bare
anthropic
bare render of tier-2-hex-bolt
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 47.1s 6009/2839t $0.1010
openai
google
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

皿穴付き L 字金具 tier 2 tier-2-l-bracket

bare
anthropic
bare render of tier-2-l-bracket
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 163.1s 9765/12312t $0.3566
bare render of tier-2-l-bracket
bare
claude sonnet 4.6
effort
effort=low low
medium
effort=high high*
max
xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
9 step 169.6s 8636/10395t $0.1818
openai
bare render of tier-2-l-bracket
bare
openai gpt 5 nano 2025-08-07
effort
· low render effort=medium medium* · high nocode
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 1062.3s 6096/116744t $0.0470
google
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

取手位置を指定したマグカップ tier 2 tier-2-offset-handle-mug

bare
anthropic
bare render of tier-2-offset-handle-mug
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 142.9s 8739/10629t $0.3094
bare render of tier-2-offset-handle-mug
bare
claude sonnet 4.6
effort
effort=low low
medium
· high* nocode
max
xhigh
thinking
· off nocode thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
9 step 429.0s 12001/22495t $0.3734
openai
bare render of tier-2-offset-handle-mug
bare
openai gpt 5.4 nano 2026-03-17
effort
effort=low low effort=medium medium* · high render
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 127.0s 9084/17117t $0.0110
no render
bare
openai gpt 5 mini 2025-08-07
effort
effort=low low · medium* render effort=high high
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 756.7s 5879/50978t $0.1034
no render
bare
openai gpt 5 nano 2025-08-07
effort
· low render · medium* render · high nocode
max
xhigh
iterpng
· i1 render · i2 render iter 3 i3
7 step 936.1s 3472/93357t $0.0375
bare render of tier-2-offset-handle-mug
bare
openai gpt 5.1 codex max
effort
low
effort=medium medium*
high
max
xhigh
iterpng
· i1 render iter 2 i2 iter 3 i3
5 step 268.6s 4534/33922t $0.3795
google
bare render of tier-2-offset-handle-mug
bare
gemini flash-lite 2.5
thinking
· off render
adaptive
thinking=dynamic dynamic*
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
6 step 26.8s 5346/5906t $0.0029
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

バット蝶番(2 枚 + ピン軸) tier 3 tier-3-butt-hinge

bare
anthropic
bare render of tier-3-butt-hinge
bare
claude opus 4.7
effort
effort=low low
medium
effort=high high* effort=max max effort=xhigh xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
11 step 248.2s 15189/18535t $0.5393
bare render of tier-3-butt-hinge
bare
claude sonnet 4.6
effort
effort=low low
medium
effort=high high*
max
xhigh
thinking
thinking=off off thinking=adaptive adaptive*
dynamic
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
9 step 240.0s 13852/15375t $0.2722
openai
no render
bare
openai gpt 5.4 nano 2026-03-17
effort
effort=low low · medium* render effort=high high
max
xhigh
iterpng
· i1 render · i2 render · i3 render
7 step 170.8s 8689/25967t $0.0162
bare render of tier-3-butt-hinge
bare
openai gpt 5 mini 2025-08-07
effort
· low render effort=medium medium* effort=high high
max
xhigh
iterpng
iter 1 i1 · i2 render iter 3 i3
7 step 1170.2s 10690/57422t $0.1175
bare render of tier-3-butt-hinge
bare
openai gpt 5 nano 2025-08-07
effort
· low render effort=medium medium* · high nocode
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 1323.7s 9229/111303t $0.0450
no render
bare
openai o4 mini 2025-04-16
effort
effort=low low · medium* render effort=high high
max
xhigh
iterpng
iter 1 i1 iter 2 i2 iter 3 i3
7 step 397.8s 7367/53331t $0.2428
google
bare render of tier-3-butt-hinge
bare
gemini flash-lite 2.5
thinking
thinking=off off
adaptive
thinking=dynamic dynamic*
iterpng
· i1 render iter 2 i2 iter 3 i3
6 step 98.3s 21249/22522t $0.0111
openai (self-hosted)
google (self-hosted)
nvidia (self-hosted)
qwen (self-hosted)

距離センサ GP2Y0D413K0F の外形モデリング(データシート図から) tier 4 tier-4-gp2y0d413k

pdf-page
anthropic
openai
google
google (self-hosted)

距離センサ GP2Y0A21YK0F の外形モデリング(データシート図から) tier 4 tier-4-gp2y0a21yk

pdf-page
anthropic
openai
google
google (self-hosted)