Help with prototype functions in c programming?

#include <math.h>

One prototype for both functions:-
void pytheta(double x, double y, double& pyth, double& theta);

Then after main:-
void pytheta(double x, double y, double& pyth, double& theta){
*pyth = sqrt(x*x+y*y);

*theta = atan(y/x);
}

the above with crash if x==0
so it is better to use
*theta = atan2(y,x);

Get the terminology straight. You're using "prototype" to mean "function".

A function is a self-contained chunk of code that can accept input in the form of arguments, performs some computations, and finally returns a value to the caller in the form of a function result.

A prototype declares the function for the caller. In C, you have to declare a name before you use it. A prototype does that for a function, giving the name of the function, the number and types of the arguments expected, and the data type of the returned result.

A function definition is where you say what the function actually does.

double hypotenuse(double x, double y); /* This is a prototype */

/* Below is a function definition: */

double hypotenuse(double x, double y)
{
return sqrt(x*x + y*y);
}

It's the function, not the prototype, that does the calculation. You need a prototype when either the function either appears later in the current source file, or in another source file altogether.

Two other bits of C terminology: "Arguments" are the actual values passed by the calling program to the function. The names x and y that stand for those arguments, in both the prototype and the function definition, are "parameters". Some other languages use "actual parameter" instead of "argument" and "formal parameter" instead of "parameter". These get mixed up a lot, and you'll hear "formal argument" instead of parameter from some, and just "argument" for everything from others.

The other bit is that before the run-up to the first ANSI standard for C, nobody used the term "prototype". The term in the docs was "function declaration", but because it looks just like the function header, without the {} braced function body, they were often just called "headers". This is how included files came to be called "header files"--they were mostly function declarations...headers.

those are not prototypes
you want the actual functions

double pyth(double,double); // is the prototype -- it just tells the compiler how to call the function and what the return type is...

double pyth(double a, double b){ // this is the function definition...
return sqrt(a*a+b*b);
}

Two "prototypes"? Not sure what you mean by that. I think you mean to use another word. Maybe you mean functions?

For the formulas, you'll need to #include <math.h>. Then you write them like this:

double r=sqrt(pow(x,2)+pow(y,2));
double theta=atan(y/x);

Of course x and y need to be defined somewhere in your code before this.