Brandon Sanderson famously defined magic systems as either soft or hard, with soft magic having no defined rules, while hard magic has known rules. When using magic in a narrative, the more the reader understands the rules of the magic, the more the author can use it to solve the protagonist's problem. The idea being that if the reader doesn't understand how magic within this fantasy world works, then it feels like the author is just creating a convenience for the characters to use to solve their problem. This can lead to a feeling that there are no stakes within the story and that the characters are never in any real danger.
Soft Magic
While soft magic can certainly exist within a setting, it can't really be used by the characters. What I mean is that soft magic works well for magic that is not controlled by the characters, but instead comes from the environment or a rare phenomenon. In any fantasy world, if a character is using magic, by its very nature, it must be a hard magic system.
The idea that characters can utilize a soft magic system is a silly one at best. If you are going to make use of something, you have to have at least some idea of how it works. Assume for a moment that there's a supernatural force in the world, and you can freely tap into it, but it has no rules at all. This force essentially is pure chaos. Each time you use it, you have no idea if you're going to get what you want, or if you're going to accidentally blow up your house. You'd never use it. The risk of catastrophic outcomes just isn't worth it. What if it's not pure chaos? What if you know that this force is only able to produce a limited range of outcomes? Well, now it has rules. That makes it hard magic.
When describing soft magic, a lot of people will point to The Lord of the Rings as a soft magic system. This isn't accurate at all. The Lord of the Rings actually uses hard magic. For instance, Gandalf remarks that he once knew all the spells in the languages of men, elves, and orcs. That tells us that in order for people in Middle Earth to cast spells, these spells must be known, and there are a limited number of spells in existence. We don't know if it's possible to experiment and create new spells, or if the spells of Middle Earth simply exist and people had to discover what works and what doesn't, but in either case, we know that there are a limited number of spells. In The Hobbit we see Gandalf ignite pinecones and hurl them at the goblins. In The Lord of the Rings Legolas suggests that Gandalf might wield a flame to melt the snow to make the passage over Caradhras easier, and Gandalf says that he can't simply create a flame. He needs something to ignite. This tells us that there are limits to these spells and Gandalf can't simply do whatever he wants. When the wolves attack the Fellowship in the night, Gandalf draws flames from their campfire to hurl at the wolves. This shows us that Gandalf can manipulate fire, but it didn't just come from nowhere, there was a source. He also says that if anyone was watching, he just announced his presence to everyone for miles around. This shows us that using magic, especially powerful displays like this, makes it easier for other magical beings to sense him.
So, while the reader doesn't get to see all of the rules of magic in The Lord of the Rings it does abide by a set of rules. You might think that "Avatar: The Last Airbender" uses a soft magic system. Nope, that's hard magic, too. There are rules. You must be born a bender, and you will only ever bend your native element. Bending comes from a natural energy flow, and is typically focused through the use of a particular martial arts style. Bending can be shut off by blocking chi flow within the body. These are all rules that exist within that world.
What about the Force? Yes, even the Force has rules. While it is possible for people within "Star Wars" to use the Force, it's typically in a limited capacity until they are trained. The Jedi spend years in training in order to properly wield the Force. The Sith require a lot less training because they're willing to accept the corruption of the Dark Side, enabling them to obtain power quickly. If the Force didn't have rules, then the Jedi wouldn't need to spend so much time training. Other practitioners, such as the Night Sisters use rituals akin to our concept of witches in order to use the Force. Again, that's a set of rules. They have different traditions, but they know that if they perform certain rituals, the Force will react in a particular way, thus, rules.
So What is Soft Magic?
The weather of Middle Earth is a good example. Sometimes the world itself responds to what's going on. An evil army of Uruk-hai march upon Helm's Deep, ready to exterminate the kingdom of Rohan. This causes it to rain. You can include elements of soft magic in a narrative by tying it to the world. The best way to utilize soft magic is to have environmental things just happen. Soft magic should never be wielded by your characters, because by its very nature, it can't be.
Is Sanderson Wrong?
No, of course not. The basic principle is sound. The more your reader understands how magic works in your world, the more natural it will feel for your characters to use it. If you solve problems in your story by using magic in a way that the reader can't justify, then your narrative begins to suffer. That doesn't mean that you can't have your character use new powers. It just means that when your characters suddenly do something new, it needs to feel natural. If characters in your story never fly, and suddenly your protagonist flies, you better have a good explanation. However, if flight has already been shown as a possibility, then your protagonist learns to fly, it feels natural. You didn't just pull it out of nowhere, you laid the ground work to establish what was possible.
I've spent quite a while talking about hard vs soft magic, but there's another aspect of magic that Sanderson codified, and that is the fact that what really makes magic interesting is its limits. If absolutely anything can eventually become possible, then you're writing a story about omnipotent gods. However, if your characters need a specific material to cast spells, or if every spell cast consumed a resource or had a rippling effect elsewhere in the world, or if you simply decide that you don't want your magic to be able to do certain things or affect certain things then you've introduced something interesting to your world. Remember, the most interesting part of your characters is their flaws. Think of magic as another character. It needs its own flaws to make it interesting.