Setup link previously down beneath user name, in new user interface, it is appear next to Help & Training link at top right after username, My Settings will replace the existing place for Setup. See screenshots below, this is only when Accessibility Mode if off for your user detail.
You can activate new setup menu from Setup | Customize | User Interface, look for Setup section and select Enable Improved Setup User Interface. Please note, this will affects all users in your organization, so you better communicate with your other admins and users before activate it. It is good practice to always try in the sandbox instance or developer instance if you do not have any sandbox instances, to get your users approval.
For experience admin, new menu will make us to re-understand where the menu move to, but after a while you will be familiar and this new menu is make sense. You will see it will be group into better order and less click to reach a menu.
Thanks for Mike Gerholdt for create and sharing the changes into a graphical presentation.
right click image above and click 'Open link in new tab' to see bigger image.