Stir together rice milk, almond milk, and condensed milk in a gallon jug. Mix in vanilla and almond extracts. Pour about 2 cups into a bowl; whisk in cinnamon until mostly combined.
Pour cinnamon mixture back into the jug; stir to combine. Add cinnamon sticks and refrigerate for at least 4 hours to overnight.
Stir horchata before serving over ice.
