Kayes had helped Bangladesh stage something of a recovery from the early loss of Tamim.
Duminy forced an edge from Mominul for 40, ending the batsman’s 69-run second wicket stand with Kayes.
Kayes followed Mominul, who hit six fours off 87 balls, when Duminy struck in his next over, trapping the left-hander leg-before for 30. Mahmudullah was given out leg-before on 18 off Steyn but the decision was overturned on review as a replay showed a big inside edge.
Steyn later broke his partnership with Rahim as the right-hander flicked a catch to Temba Bavuma at short midwicket after reaching 35.
Rahim had been looking solid until occasional left-arm spinner Elgar spun one sharply to take his glove and the ball carried to wicketkeeper Dane Vilas.
Elgar then pouched Liton Das at short midwicket as Duminy claimed his third wicket.
Shakib Al Hasan edged Morne Morkel to Elgar at gully for 35 before Steyn bowled Mohammad Shahid in the last ball of the day to complete Bangladesh’s misery.
South Africa handed wicketkeeper-batsman Vilas a Test debut in the match, replacing Quinton de Kock. The 30-year-old became South Africa’s sixth new cap in the last 18 months. Bangladesh recalled Nasir in place of left-arm spinner Taijul Islam. The first Test ended in a draw after rain washed out the last two days in Chittagong.
Score Board
Bangladesh won toss
Bangladesh 1st innings
Tamim Iqbal c Amla b Steyn 6
Imrul Kayes lbw b Duminy 30
Mominul Haque c Vilas b Duminy 40
Mahmudullah c Bavuma b Steyn 35
*Mushfiqur Rahim c Vilas b Elgar 65
Shakib Al Hasan c Elgar b M Morkel 35
†Liton Das c Elgar b Duminy 3
Nasir Hossain not out 13
Mohammad Shahid b Steyn 1
Extras (b 5, lb 11, nb 2) 18
Total (8 wickets; 88.1 overs) 246
To bat: Mustafizur Rahman, Jubair Hossain
Fall: 1-12, 2-81, 3-86, 4-180, 5-215, 6-220, 7-245, 8-246
Bowling: Steyn 16.1-4-30-3; Philander 11-2-25-0(2nb); M Morkel 14-2-45-1; Harmer 23-3-76-0; van Zyl 2-1-5-0; Elgar 7-0-22-1; Duminy 15-4-27-3
South Africa team
D Elgar, S van Zyl, F du Plessis, *H M Amla, T Bavuma, J P Duminy, †D J Vilas, V D Philander, D W Steyn, S R Harmer, M Morkel
Test debut; DJ Vilas (South Africa)
Umpires: R A Kettleborough (England) and P R Reiffel (Australia). TV umpire: J S Wilson (West Indies. Match referee: B C Broad (England)