Ross Wallace put Wednesday ahead after 27 minutes and set up Lucas Joao for the second goal shortly before half-time, with Sam Hutchinson completing victory early in the second period.
It was Wednesday’s second Premier League upset in this season’s competition, following their third-round win at Newcastle United.
Arsene Wenger fielded a relatively weakened squad and also lost Alex Oxlade-Chamberlain and Theo Walcott to injury in the first half.
Asides bringing Arsenal’s run of four successive victories to an end, the loss also means Arsene Wenger’s wait for a first League Cup of his lengthy reign goes on.
The Gunners have not won the competition since 1993, when they were victors over the Owls at Wembley.
Chelsea needed a late Loic Remy equalizer to force their game against Stoke City into extra time but eventually lost 5-4 on penalties in another upset on the night.
With nine well taken penalties by both teams, Chelsea’s hope were dashed as Eden Hazard saw his final spot kick saved in a spectacular manner by Stoke’s Asmir Begovic.
Everton beat Norwich City 4-3 on penalties while Hull City also defeated Leicester City 5-4 on penalties after both games ended 1-1 at regulation time.