Wayne Swan's warning about One Nation marks a notable shift in tone from Prime Minister Anthony Albanese.