Germany performed the gesture as they lined up for a team photo before their shock 2-1 loss against Japan in their World Cup opener, with their coach Hansi Flick saying it was a way of showing "FIFA is muzzling us."
Belgium, Germany and five other European teams abandoned the anti-discrimination OneLove initiative earlier this week after players were threatened with sanctions if they wore the rainbow armband.
Speaking after his side's 1-0 win against Canada on Wednesday, however, Hazard said it would have been better if Flick's side had not made a stand and won their game.
"Yes but afterwards, they lost the match," Hazard said, when asked for his thoughts on the gesture. "They would have been better off not doing it and winning.
"We're here to play football. I'm not here to pass on a political message, there are people who are better placed for that.
"We want to be concentrated on the football."
FIFA threatened team captains with a yellow card if they wore the OneLove armband, but Hazard admitted he might have reconsidered his position in different circumstances.
"I'm not at ease talking about it because I'm here to play football," Hazard said.
"We were kind of forbidden from... I didn't want to start the match with a yellow card, it would have been annoying for the rest of the tournament.
"If I were to redo it, maybe I would wear it again."
In a news conference on Thursday, Wales captain Gareth Bale said his side had been displeased with the decision to abandon the OneLove initiative while adding that gestures could backfire if teams lost.
"We weren't too happy about not being able to wear it," Bale said. "I know people said 'Oh, I should have worn it' but I would've been sent off after 25 minutes, so of course we support it but we're here to play football at the same time.
"In terms of doing something else, I guess when teams try and do something else and the result doesn't go the right way, then they get criticised for not concentrating on the football.
"For us, I think now the tournament's started we really need to concentrate on the football ourselves, but outside of the game, if there's anything that we can do to raise awareness or support, then we'll definitely do that."
Homosexuality is illegal in Qatar while it was described as "damage in the mind" by one tournament ambassador before the finals began in the Gulf state this month.
The tournament has proved highly controversial, with FIFA president Gianni Infantino giving an extraordinary opening address in which he criticised European nations' view of the finals.
Before the World Cup, Infantino wrote a letter to the 32 nations participating to ask them to "focus on the football" rather than getting dragged into political debates.