There are two options for embedding videos, depending on how your page is setup. The first one is using the Video Block Layout

Scroll to your "Layouts" and click "Add Row"

Paste the video URL (taken from YouTube/Vimeo etc) in the "Enter URL" field.

The video will then be shown in the block.

The other option is to embed the video in a text block - this is useful if you're looking to include a video inside another layout tool, e.g. - an accordion

On Your WordPress page editor, click "Text" at the top of your text block

Paste the embed code into the block, the click "Visual"

Your video will then be embedded in your text block

Although this guide shows YouTube videos specifically, any other hosting side such as Vimeo etc will have similar options to embed.

How to Embed a Video on Your Website