As the authorities insisted that the Village was only experiencing teething problems and would be ready shortly, questions arose over alleged lack of legal protection for the approximately 600 people drafted to make emergency repairs.
The ministry of labor said in a statement that inspectors had discovered "signs of informality in contracts" and working days of "up to 23 hours."
Fines may be imposed, the ministry said.
Inspector Hercules Terra told the G1 news site that the absence of proper contracts for the workers meant that "if a worker suffers any sort of accident or is killed, then the family will not have this guarantee."
The Rio 2016 organizing committee did not immediately respond to a request for comment.
Earlier, Rio Mayor Eduardo Paes and the Australian team made peace following its harsh complaints about conditions in the Village.
Paes kissed Australia delegation chief Kitty Chiller on the cheek, handed her a symbolic key to the city and apologized for the construction hiccups, which included blocked toilets, dangerous wiring and leaks.
"I saw the things. This was the worst building. I recognize the problems you faced," a repentant Paes said.
Paes' speech was a far cry from his apparently combative reaction Sunday after Chiller lambasted the accommodations as the worst she'd seen in five Olympics.
The mayor's quip that a kangaroo should be sent to make the Australians feel better fell flat, being widely interpreted as a jibe and quickly surfacing on Twitter as #kangaroogate.
Paes explained Wednesday he'd not meant any harm and offered "a formal apology, almost a diplomatic thing."
Flanked by Australian athletes, Chiller praised Brazil for the "passion and the commitment" in carrying out emergency repairs -- and gave Paes a stuffed kangaroo.
About half of the buildings in the 31-building complex, which will house some 19,000 team members from around the world, have had issues.
But International Olympic Committee chief Thomas Bach, who arrived in Rio on Wednesday, said that "the remaining ones will be solved in the next 24 or 48 hours."
Asked later how much the rush repair job was costing, Paes responded: "I don't know."
- Brazilians gloomy over Olympics -
The Village's problems are another embarrassment for Brazil, which is struggling to show it can cope with the pressure during a severe recession and political crisis.
Organizers are already facing questions over low ticket sales, public apathy, fears over the Zika virus, and a spike in street crime as police complain of lack of resources.
A poll published in Sao Paulo's Estadao newspaper found that 60 percent of Brazilians believe the Olympics will bring more bad than good to the country.
The Ibope poll found only 32 percent of Brazilians expect the first Games ever held in South America to be more positive than negative.
This split showed a darker mood than on the eve of the last major sporting event hosted by Brazil, the 2014 football World Cup. Then, 40 percent expected more negative outcomes compared to 43 percent who were optimistic.
Paes brushed off the poll, telling Estadao that "the city of Rio is already benefiting from the legacy, the population is already benefiting."
When Brazil won the Rio hosting rights back in 2009, the economy was booming and then president Luiz Inacio Lula da Silva was immensely popular.
Today Brazil is in economic retreat, while Lula faces serious corruption allegations and his handpicked successor Dilma Rousseff risks being removed from office in an impeachment vote shortly after the Games end.
Amnesty International on Wednesday highlighted bloody police violence in Rio by laying 40 body bags in front of the Olympic organizing committee headquarters -- each representing people killed by city officers in May alone.
"Our main worry is that there is an escalation of police violence as we get closer to the Games," said Amnesty expert Renata Neder.