Photon

To safety remove dynamic objects it's recommended to use LazyDestroy helper script. You can create new extension to make removal easier. Here is an example for Fusion:

public static class NetworkRunnerExtension {
	public static void SafeDespawn(this NetworkRunner runner, NetworkObject obj) {
		obj.gameObject.LazyDestroy(() => runner.Despawn(obj));
	}
}

Then you can call it like that:

Runner.SafeDespawn(Object);

That code will hide object, so combinables will be removed from baked mesh, and then call runner.Despawn(obj)