Police are now negotiating with the man, who said he acted in "revenge for Palestinian children" and French military operations abroad.
Two police officers were injured in exchanges of fire during the raid and the suspect's brother is under arrest.
The man's mother, who is Algerian, has been brought to the scene, but Mr Gueant said she had refused to become involved as "she had little influence on him".
The minister said the suspect had made several visits to Afghanistan and Pakistan.
"He claims to be a mujahideen and to belong to al-Qaeda," Mr Gueant said.
"He wanted revenge for the Palestinian children and he also wanted to take revenge on the French army because of its foreign interventions."
The man shot at the door after police arrived, Mr Gueant said, injuring one officer in the knee and "lightly injuring" another.
One official told Agence France-Presse the suspect had been "in the sights" of France's intelligence agency after the first two attacks, after which police had brought in more "crucial evidence".
French media have linked the suspect to a group called Forsane Alizza (Knights of Pride) that was banned by Mr Gueant last month.
They also say the suspect had earlier been arrested in Kandahar, Afghanistan, for unspecified but not terrorist-related criminal acts.
The BBC's Hugh Schofield in Paris says investigators report the suspect's first name as Mohamed and that he was identified because of an email message sent to his first victim about buying a scooter.
The message, sent from the suspect's brother's account, set up an appointment at which the soldier was killed, sources told AFP.
The man had also sought out a garage in Toulouse to have his Yamaha scooter repainted after the first two attacks. A scooter was used in all the attacks.
Our correspondent says the house in Toulouse is a five-storey block of flats and the man is on the ground or first floor.
Police wearing helmets and flak jackets have cordoned off the area and prosecutors say other operations are under way to track down possible accomplices.
The brother was reportedly arrested in another part of Toulouse and a second brother has attended a police station, French media say.
A huge manhunt had been launched after Monday's shooting at a Jewish school that left four people dead, and the killing of three soldiers in two incidents last week.

Memorial services

The funerals of the rabbi and three children killed on Monday are due in Jerusalem in the coming hours.
Israeli police said they expected thousands of people to attend.
The attacker gunned down Jonathan Sandler, a 30-year-old rabbi and teacher of religion, his two young sons Arieh and Gabriel and then - at point blank range - the head teacher's daughter, seven-year-old Myriam Monsonego, in Monday's attack at a Jewish school in Toulouse.
Their bodies were carried out of Ozar Hatorah school on Tuesday in two black hearses and taken to a nearby airport.
A military jet then flew them to Paris, from where they were placed on a commercial flight to Tel Aviv. They have now arrived in Israel.
French Foreign Minister Alain Juppe has accompanied the relatives of the dead to the funerals in Jerusalem.
Also on Wednesday, French President Nicolas Sarkozy is to attend a memorial service for the three soldiers killed in the two attacks last week.
All three were of North African descent. Another soldier from the French overseas region of Guadeloupe was left critically ill.
Socialist presidential candidate Francois Hollande and Marine Le Pen, the leader of the far-right Front National, will attend the memorial service in Montauban.
After Wednesday's raid took place, Ms Le Pen said the "fundamentalist threat has been underestimated" in France.