Error
W1204 00:25:55.768000 4063995 site-packages/torch/_export/__init__.py:64] +============================+
W1204 00:25:55.769000 4063995 site-packages/torch/_export/__init__.py:65] | !!! WARNING !!! |
W1204 00:25:55.770000 4063995 site-packages/torch/_export/__init__.py:66] +============================+
W1204 00:25:55.770000 4063995 site-packages/torch/_export/__init__.py:67] capture_pre_autograd_graph() is deprecated and doesn't provide any function guarantee moving forward.
W1204 00:25:55.771000 4063995 site-packages/torch/_export/__init__.py:68] Please switch to use torch.export.export_for_training instead.
class GraphModule(torch.nn.Module):
def forward(self, input_ids):
arg0: "i64[2, 9]";
arg0, = fx_pytree.tree_flatten_spec(([input_ids], {}), self._in_spec)
arg0_1 = arg0
Solution
Solution is to look into the implementation of export_for_training and update it to handle tied weights correctly. This could involve adding specific code to account for the tied weights in the prepare_pt2e model, such as properly propagating updates to the tied parameters during the training export process. Alternatively, if it's feasible, a custom version of the export_for_training function could be created that caters specifically to models with tied weights like the prepare_pt2e model.