I thought it would be good to help others in terms of my learning journey when working with partner notebooks and Microsoft fabric.
In today’s blog post, I am going to show you how to rename a column. In my experience this came up because I had a column name which had a forward slash “/” in it which caused the loading of the data for the table to fail because this is a reserved character.
If you find this blog post interesting and would like to get more blogs on how to manipulate data using Python notebooks, please let me know in the comments section. I do have quite a few other ideas!
I think it is also good to know how to rename a column because there are some instances when column name is not what is exactly needed to be loaded into Lakehouse table.
In my example I loaded a CSV table as shown below.
As far as I am aware currently column names cannot have a space in them for the column name.
As per my example above I have a column called “Order Number”.
Below is the python code I used to then rename the column from “Order Number” to “OrderNumber”
#Rename Column so that it will load correctly, by removing the space from 'Order Number'
from pyspark.sql import SparkSession
# Rename the 'Order Number' column to 'OrderNumber'
df = df.withColumnRenamed("Order Number", "OrderNumber")
#Show the changes of the column rename
display(df)
Now when I ran the above code, I got the following result below with the column renamed.
In this blog post I have shown you how to rename the column name when working with a Python data frame.
Once again, if you find this blog post interesting and would like to get more blogs on how to manipulate data using Python notebooks, please let me know in the comments section. I do have quite a few other ideas!
Thanks for reading!
Be the first to comment