Modern day computer security relies heavily on cryptography as a means to protect the data that we have become increasingly reliant on. The main research in computer security domain is how to enhance the speed of RSA algorithm. The computing capability of Graphic Processing Unit as a coprocessor of the CPU can leverage massive parallelism. This paper presents a novel algorithm for calculating modulo value that can process large power of numbers which otherwise are not supported by built-in data types. First the traditional algorithm is studied. Secondly, the parallelized RSA algorithm is designed using CUDA framework. Thirdly, the designed algorithm is realized for small prime numbers and large prime number. As a result the main fundamental problem of RSA algorithm such as speed and use of poor or small prime numbers that has led to significant security holes, despite the RSA algorithm’s mathematical soundness can be alleviated by this algorithm. Public key cryptography is a fundamental and widely used technology around the world. Most public key algorithms are based on modular arithmetic including RSA, Public key encryption and decryption is computationally heavy because a lot of modular multiplications with very large numbers are needed to perform these tasks. Therefore public key algorithm is known to be much slower than symmetric key algorithms. Recently the field of using GPUs for general purpose computing has become more widespread. Many computational problems have gained a significant performance increase by using the highly parallel properties of the GPU. CUDA is a framework which makes these kinds of implementations more available to the general public of programmers.