Explicit Path Question · Learn Microsoft Fabric

May 28 (edited) in Technical

Explicit Path Question

Good morning,

I have been following along with the Spark Tutorial in Microsoft Fabric and ran into an interesting issue that I have yet to understand. After leaving my office Friday I paused the lesson after running the line of code:

df = spark.read.csv('Files/property-sales-extended.csv', header=True, inferSchema=True)

df.show()

The code ran fine on Friday and did again this morning.

After loading another file the code failed:

df = spark.read.csv('Files/property-sales-missing.csv', header=True, inferSchema=True)

df.show()

The error message was that the path was not valid.

Same LakeHouse, same folder, same everything as far as I could tell. I solved the issue by changing the call to explicitly include the name of the LakeHouse folder:

df = spark.read.csv('Files/csv/property-sales-missing.csv', header=True, inferSchema=True)

df.show()

The ABFS path also works. Can someone help me understand the reason for this?

Thanks,

Steve Malcolm

4 comments