The workers, who are represented by the Cement and Concrete Workers of New York City, have been in negations with management represented by the Cement League ever since their contract expired on July 1, the Journal reported.
If negotiations falter the walk-off could lead to a strike, those close to the negotiations told the Journal.
The walk-off will reportedly delay construction on the World Trade Center's transit hub but will not affect completion of the 9/11 Memorial.
Construction on Tower 1 will reportedly be able continue for about a week without disruptions.
The memorial is still expected to open by the 10th anniversary of 9/11, a Port Authority spokesman told the Journal.