AI has been transforming stock markets through faster data analysis, large scale market surveillance, market predictions, improved trade execution through data analysis softwares, research tools, broker apps, advisory platforms, quick updates, notifications and automated trading systems. By processing large volumes of prices, share volumes, stock market filings, business innovations, investable entities, market growth news, earnings calls, market sentiments, etc. AI tools predict future directions, volatility, risks & trend strength and ultimately give trade guidance to traders and investors. Though it has a lot of advanced features, it is still not accurate and it possesses the risk of data loss or misuse as well. Due to algorithm manipulation, investors & traders can be forced to make wrong decisions leading to huge losses and gains to some ill minded people. Distorting the market behavior, triggering crowding and instability are also some negative impacts. Yet India has not been able to develop a specific AI law for stock markets and relies solely upon toothless AI guidelines & regulations on this issue. This paper suggests that toothless rules and regulations for AI use disclosure and accountability are not enough and a strong specific law on AI use in stock markets is much needed along with strengthening the powers and role of SEBI in regulating these advanced technologies in the Indian stock markets.