Preposition ends of clause?

The answer from Radioman is not completely accurate. Dropping the final preposition is incorrect if the preposition is required. In that case, the sentence can be structured differently to avoid the situation.

In the case of the original poster, there is no reason to include the preposition "with" since it is not a required preposition. The sentence only means that you are saving money, and that allows you to start a business.

If the sentence is "I need to save money because it will be the principle that I start my business with", the "with" is required and cannot be dropped. That is, you cannot write:
I need to save money because it will be the principle that I start my business.

You can choose to rewrite the sentence by placing the preposition before the relative pronoun that:
I need to save money because it will be the principle with which I start my business. (You need to change that to which since you have a preposition.)

It is not forbidden in English for the preposition to be hanging at the end of a sentence. In some cases, particular in phrasal verbs, it is preferred to leave the preposition after the verb because it makes the sentence easier to read.

For example: "There is no one whom/that I can count on" sounds a lot more natural than "There is no one on which/whom I can count."

Yea...a very text book example of never end a sentence with any preposition, it sound like you're putting someone on a spring cord and suspend him in mid air and he never get to touch the ground..(lol)

By reading: "I will also save money to start my business with" you would inevitably notice that a clause before this sentence is missing because with must connect by something to make the sentence complete.

Perhaps :
With the start of my business, I first have to save some money.
(first) = missing link.