Vortex generators are simple aerodynamics devices used mainly to keep the airflow attached over a surface and delay flow separation. These are small, fin or vane-like structures attached perpendicular to the surface over which air is supposed to flow and aligned against the direction of airflow at certain angles.
You might most commonly notice them near the leading edge of the wings of an aircraft and look somewhat similar to this:
These devices also find applications in the rotor blades of turbines, and even in cars to improve the aerodynamics by reducing overall drag.
Taking the example of a typical aircraft wing, the cross-sectional shape is in the form of an airfoil. This airfoil shape is what allows the aircraft wing to generate a lift force for flight. Airfoils are effective in generating this force only when the air remains attached to its surface. If we look at a typical airfoil, here are the main terminologies associated with it:
The angle of attack, as we can see clearly from the diagram, is the angle between the airfoil chord line and the relative direction of airflow. As the angle of attack is increased, the lift force increases. However, this increase in lift force is only up to a certain angle of attack called the stall angle, after which the lift force will start decreasing as the angle of attack increases.
When air flows over the upper surface of the airfoil, the pressure decreases until it reaches the center of lift - about 25% down the airfoil's chord. Then, pressure starts to increase again, so the air moves from an area of low pressure to higher pressure. This is called an ‘adverse pressure gradient’. As the airflow moves towards high pressure from low pressure, it loses energy. Eventually, when it runs out of energy, the airflow separates from the wing.
At higher angle of attack, the normally laminar air flow tends to detach from the surface of the airfoil near its trailing edge region as it loses energy and form vortices. Control surfaces like ailerons, rudder and elevator are located in the trailing edge region. Hence, as the air flow is detached from the trailing edge region, the control surfaces may become ineffective at higher angle of attack and it might become very difficult to try to recover an aircraft from a possible stall.
The air just near the surface of the airfoil forms a boundary layer, which is a thin layer of air that loses energy due to friction. A laminar boundary layer is more smooth and hence, has less skin friction drag (drag over the airfoil surface due to friction).
The air above the boundary layer isn't affected by skin friction, so it has more energy than the air in the boundary layer. Pulling in some of that free-stream air into the boundary layer could add energy and delay the boundary layer's separation. This is where the vortex generators come into play.
Vortex generators act like tiny wings and create mini wingtip vortices, which spiral through the boundary layer and free-stream airflow. These vortices mix the high-energy free-stream air into the lower energy boundary layer, allowing the airflow in the boundary layer to withstand the adverse pressure gradient longer. This allows for the aircraft to operate effectively even at higher angle of attack and prevent stalling.
Essentially, vortex generators convert the laminar boundary layer over an airfoil into a turbulent boundary layer, which despite having more skin friction drag, offers the benefit of keeping the airflow attached to the airfoil surface and makes it more effective at higher angle of attack. The below images will make it more clearer as to how vortex generators work:
The reference links that I used for this answer might provide a better insight into these devices.
Links for images and reference: