North Korea often makes threats and occasionally tests missiles or other weapons. For South Koreans, hearing news of North Korean threats or missile tests is just another part of the day. South Korea is barely influenced by North Korean threats. This is because the same situation has been going on for the past 50 years and nothing serious has ever broken out. Neighboring nations and allies of Korea such as Japan, Russia, China, and the US have an interest in maintaining peace in the Korean peninsula and have played great roles in doing so. An attack by North Korea would mean the complete destruction of North Korea by South Korea and its allies so a war or large-scale conflict are not likely to happen.
Approximately 1.9 million foreigners are residing in Korea as of the beginning of the year 2016. Most of these foreigners are living happily without much concern about threats from Korea. Foreigners who have been in Korea for a few years are not affected by news of threats anymore.
If you are still worried about your safety, you can visit your Embassy and register yourself as a teacher working in Korea. Your embassy will then notify you about any situations that occur in Korea and if there is a reason to leave Korea. If there is a situation that requires evacuation, your embassy will guide you to an evacuation point.