+ jhO:Rt^RIt^RIHt^RIHtHtHtHt^RI t !RR4t RRlt ] R8Xd^RI t R t]!R 4]!R4]!R ] R 24R R/t] P"!]]^ R7t]P&^8Xd] !]P(]4t] !]P(]4t]P/4]!R4]!R4]!]P144]!R4]!R4]!]P344R#]!R]P& 24R#R#)zp Structured Property Facts Extractor Extracts structured data from property pages for high-quality GPT analysis N) BeautifulSoup)DictOptionalListAnyc2a]tRt^ toRtV3RlRltV3RlRltV3RlRltV3RlR ltV3R lR lt V3R lR lt V3RlRlt V3RlRlt V3RlRlt V3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltR tVtR!#)"PropertyFactsExtractorz+Extract structured facts from property HTMLc&<V^8dQhRS[RS[/#) html_contenturlstr)format __classdict__s"^/Users/jonathan/Documents/Zakelijk/ClaudeOS/03_Lab/farmmatch/scraper/extract_property_facts.py __annotate__#PropertyFactsExtractor.__annotate__sSscB\VR4VnW n/VnR#)z html.parserN)rsoupr facts)selfr r s&&&r__init__PropertyFactsExtractor.__init__s!, >  rc6<V^8dQhRS[S[S[3,/#r returnrrr)rrs"rrrsT#s(^rc RVPRVP4RVP4RVP4RVP 4RVP 4RVP 4RVP4R VP4R VP4R VP4R VP4/ Vn VP#) z3Extract all structured facts from the property pager title descriptionpricelocationproperty_details land_detailsbuilding_details amenitiesfeatures energy_infokey_highlights) r _extract_title_extract_description_extract_price_extract_location_extract_property_details_extract_land_details_extract_building_details_extract_amenities_extract_features_extract_energy_info_extract_key_highlightsrrs&r extract_all"PropertyFactsExtractor.extract_alls 488 T((* 4446 T((* ..0  > > @ D668  > > @ 002 ..0 4446 d::<   zzrc <V^8dQhRS[/#rr )rrs"rrr%s;;;rc pVPPR4pV'dVPRR7#R#)zExtract property titleh1Tstrip)rfindget_text)rr s& rr+%PropertyFactsExtractor._extract_title%s, t$-2u~~D~)::rc <V^8dQhRS[/#rr )rrs"rrr*s - -c -rc ^VPPRRR7p.pVR,FjpVPRR7pV'gK\V4^28gK1\P !RRV\P R 7pVPV4Kl R PV4R ,#) z+Extract main property description (cleaned)divzlisting-section-contentclass_:Nr NTr<z7(Cookie|Privacy|Terms|Contact us|Show more|Lees meer).*r>)flags :NiN) rfind_allr@lenresub IGNORECASEappendjoin)rsections descriptionssectiontexts& rr,+PropertyFactsExtractor._extract_description*s99%%e4M%N ||G##$#/DtD BvvXZ\^bjljwjwx##D) $xx %e,,rc0<V^8dQhRS[S[,/#r)rint)rrs"rrr9s## #rc  VPPRRR/4pV'deVPRR7pRP\ \ P V44pV'd"\V4pRTu;8:d R8:dV#MVPP4p.R OpVFp\P!Wu\P4pV'gK2RP\ \ P VP^444pV'gKt\V4pRTu;8:dR8:d MKVu#K R # L;i K;i) zExtract price as integerspanitempropr"Tr<r>'iN)u+€\s*(\d{1,3}(?:[.,]\d{3})*(?:[.,]\d{2})?)zEUR\s*(\d{1,3}(?:[.,]\d{3})*)u(Price:?\s*€?\s*(\d{1,3}(?:[.,]\d{3})*)) rr?r@rOfilterrisdigitrVrKsearchrMgroup) r price_span price_textdigitsr"rSpatternspatternmatchs & rr-%PropertyFactsExtractor._extract_price9sYY^^FZ,AB #,,4,8JWWVCKK<=FKE11$ 2 yy!!#   GIIgR]];Eu U[[^!DE6 #F  E5X55#(L6 /&s$+EE E>EE EcF<V^8dQhRS[S[S[S[,3,/#rrrr)rrs"rrr^s"   4Xc](:#;  rc  VPPRRR7pV'dVPRR7MRpVPR4Uu.uFq3P 4NK ppRTR \ V4^8d V^,MR R \ V4^8d V^,MR R \ V4^8d VR ,/#R /#uupi)zExtract location detailsrDz item-locationrETr<r>,fullcityNregioncountry)rr?r@splitr=rJ)r location_div location_textppartss& rr.(PropertyFactsExtractor._extract_location^syy~~eO~D =I --D-9r %2$7$7$<=$s B=c6<V^8dQhRS[S[S[3,/#rr)rrs"rrrmsFF4S>Frc VPPRRR7pRRRRRRRRR RR RR RR R/pV'gV#VP4p\P!R V\P 4pV'd;\ V^,4VR&\V4^8d\ V^,4VR&\P!RV\P 4pV'd\ VP^44VR&\P!RV\P 4pV'd@\VP^4PRR44p\ VR,4VR&\P!RV\P 4pV'd\ VP^44VR&\P!RV\P 4p V 'd\ V P^44VR &\P!RV\P 4p V 'd\ V P^44VR &.ROp V FRp \P!RV ,R,V\P 4'gK?V P4VR &M \P!RV\P 4p V 'd2\ V P^44pRTu;8:dR8:d MV#WR &V#)z.Extract core property metrics from 'areas' divrDareasrE total_area_m2Nliving_area_m2 land_area_m2bedrooms bathroomsrooms property_type year_builtu(\d{1,6})\s*(?:m²|m2|sq\s*m)u>(?:plot|land|terrain|grond)[:\s]*(\d{1,8})\s*(?:m²|m2|sq\s*m)z*(\d{1,4}(?:[.,]\d{1,2})?)\s*(?:ha|hectare)ri.rZz.(\d{1,2})\s*(?:bedroom|chambre|slaapkamer|bed)z4(\d{1,2})\s*(?:bathroom|salle de bain|badkamer|bath)u'(\d{1,2})\s*(?:room|pièce|kamer|piece)\bz+(?:built|constructed|bouwjaar)[:\s]*(\d{4})i@i) farmhousevillahousecottagemanorestatemasferme boerderijuchâteaucastle) rr?r@rKfindallrMrVrJr]r^floatreplacer )r areas_divdetailsrS area_matches land_match hectare_matchha bedroom_matchbathroom_match rooms_matchtypes prop_type year_matchyears& rr/0PropertyFactsExtractor._extract_property_detailsmsIINN5N9 T d D   T T $  N!!#zz"BD"--X '*<?';GO $< 1$,/ Q,@()YY`bfhjhuhuv &)**:*:1*=&>GN # "OQUWYWdWde }**1-55c3?@B&)"u*oGN # "SUY[][h[hi "%m&9&9!&<"=GJ #Z\`bdbobop #&~';';A'>#?GK ii JDRTR_R_` ";#4#4Q#78GG DIyy*U2D"--HH+4??+<( YYMtUWUbUbc z''*+Dt#t#)- %rc6<V^8dQhRS[S[S[3,/#rr)rrs"rrrs==tCH~=rc "VPP4pRRRRRRRRRRRRRRR RR RR RR RR R/ pR.ROR.ROR.ROR.ROR.ROR.ROR .ROR .ROR .RO/ pVP4F^wrEVFCp\P!RV,R,V\P 4'gK?RW$&M W$,eKZRW$&K` \ VR,VR,VR,VR,.4VR&\P!RV\P 4'dRVR &MRVR &.ROpVFSp\P!RV,R,V\P 4'gK?VP4VR &V# V#)zBExtract land-specific details (important for regenerative farming)has_water_sourceNhas_well has_springhas_pondhas_river_access has_forest has_orchard has_vineyard has_pasturehas_arable_landirrigation_available soil_typerTFz-\b(irrigation|irrigated|arrosage|irrigatie)\b)wellpuitswaterputbron)springsourcer)ponduétangvijverlakelacmeer)riverurivièrerivierstreamruisseaubeek)forestwoodlanduforêtbostreesarbresbomen)orchardverger boomgaardz fruit trees)vineyardvignoble wijngaardvines)pastureprairieweidemeadowgrazing)arable cultivablefarmland agricultural) clayloamsandyargilelimonsableuxkleileemzand)rr@itemsrKr]rManyr ) rrSrwater_keywordskeykeywordskeyword soil_typessoils & rr0,PropertyFactsExtractor._extract_land_detailssyy!!#   $   $ 4 D 4 t "D   " = 6 K  ] ] L J O S  ,113MC#99UW_u4dBMMJJ#'GL$|#$ 4'* J  L ! J  & ' + ' "# 99EtR]] [ [.2G* +.3G* +e Dyy-tR]]CC'+zz| $  rc6<V^8dQhRS[S[S[3,/#rr)rrs"rrrs..4S>.rc zVPP4pRRRRRRRRRRRRRRR RR RR R/ pR.ROR.RORR R .R.ROR.ROR.ROR.RO/pVP4F^wrEVFCp\P!RV,R,V\P 4'gK?RW$&M W$,eKZRW$&K` \P!RV\P 4'd RVR &RVR &V#\P!RV\P 4'd RVR &RVR &V#RVR &RVR &V#)zFExtract building-specific details (for workshops, guest accommodation)has_barnN has_stable has_garage has_workshophas_outbuildinghas_separate_apartmenthas_guest_housetotal_buildingsrenovation_needed move_in_readygaragecarportrTFu8\b(renovated|restored|modernized|rénové|gerenoveerd)\buJ\b(renovation needed|to renovate|à rénover|te renoveren|work required)\b)barngrangeschuurhangar)stableuécuriestal)workshopatelier werkplaatsshed) outbuildingu dépendance bijgebouwannexe) apartmentflat appartementstudio) guest houseugîtegastenverblijfr)rr@rrKr]rM)rrSrbuilding_keywordsrrrs& rr10PropertyFactsExtractor._extract_building_detailsssyy!!#  $ $ D t $d t t  T   > 7 8Y/ I T $&T T /446MC#99UW_u4dBMMJJ#'GL$|#$ 7 99PRVXZXeXe f f'+GO $+0G' (YYdfjlnlyly z z+/G' (',GO $ ,0G' ('+GO $rc0<V^8dQhRS[S[,/#rrr)rrs"rrr$s$$DI$rc .pVPP4p.RNRNRNRNRNRNRNRNR NR NR NR NR NRNRNRNRNRNRNRNRNRNRNRNRNRNRNRNRNRNRNR NR!NR"NR#NR$NR%NR&NR'NR(NR)NR*NR+NR,NpVF^p\P!R-V,R-,V\P4'gK?VP VP 44K` \\V44#).zExtract amenities/features listinternetwififiber broadbandheatingzcentral heating chauffage verwarmingzair conditioning climatisationaircoz solar panelszpanneaux solaires zonnepanelenz heat pumpupompe à chaleur warmtepomp fireplaceu cheminéez open haardpoolz swimming poolpiscinezwembadterraceterrasseterrasbalconybalconbalkon electricityelectricu électricité elektriciteitz water mainsz town waterz eau courante leidingwaterz septic tankzfosse septiquesepticz well waterz spring waterzrainwater harvestingr) rr@rKr]rMrNr listset)rr'rS amenity_listamenitys& rr2)PropertyFactsExtractor._extract_amenities$s& yy!!#   ' )4   ) +6 8D    !0  29    0  2@    ,  .:    % '3   $ &/ 1:   " $,     "*   & (7 9H   ( *8 :H   , .6   ) +A $$Gyy50$ FF  1$C N##rc0<V^8dQhRS[S[,/#rr)rrs"rrrAs49rc .pVPP4p.ROpVF^p\P!RV,R,V\P4'gK?VP VP 44K` V#)zExtract special featuresr)zpanoramic viewz mountain viewzsea viewz valley viewprivatesecludedisolatedquietpeacefulz south facingsunnybrightstone traditional authentic characterorganic permaculture biodynamiczplanning permissionzbuilding permitzpermis de construire)rr@rKr]rMrNr )rr(rS feature_listfeatures& rr3(PropertyFactsExtractor._extract_featuresAseyy!!# $Gyy50$ FF 0$rcF<V^8dQhRS[S[S[S[,3,/#rrg)rrs"rrrUs"d3 +=&>rc VPP4pRRRR/p\P!RV\P4pV'd#VP ^4P 4VR&\P!RV\P4pV'd#VP ^4P 4VR&V#)zExtract energy performance data energy_ratingN dpe_scorez,Energy\s*(?:rating|class|label)[:\s]*([A-G])zDPE[:\s]*([A-G]|\d{1,3}))rr@rKr]rMr^upper)rrSinfo rating_match dpe_matchs& rr4+PropertyFactsExtractor._extract_energy_infoUsyy!!# T   yy!PRVXZXeXef $0$6$6q$9$?$?$AD !II94O  ) 2 8 8 :D  rc0<V^8dQhRS[S[,/#rr)rrs"rrrjs  c rc P.pVPPRR.\P!R4R7pVFjpVPR4pVR,FIpVP RR7pV'gK\ V4^8gK1VP VR ,4KK Kl V#) z3Extract key selling points from structured sectionsulolzfeature|highlight|keyrEliN NTr<:NdN)rrIrKcompiler@rJrN)r highlights key_featuresrFritemrSs& rr5.PropertyFactsExtractor._extract_key_highlightsjs yy))4,rzzJa?b)c BKK%Ec }}4}04CIM%%d4j1#rc <V^8dQhRS[/#rr )rrs"rrrysDDDrc H\P!VPR^R7#)zConvert facts to clean JSONF) ensure_asciiindent)jsondumpsrr6s&rto_jsonPropertyFactsExtractor.to_jsonyszz$**5CCrc <V^8dQhRS[/#rr )rrs"rrr}s` ` ` rc .pVPPR4'd&VPRVPR, 24VPPR/4PR4'd-VPRVPR,R, 24VPPR4'd'VPRVPR,R 24VPR 4VPPR /4pVPR 4'dVPR VR , 24VPR 4'dVPRVR , R24VPR4'dVPRVR, R24VPR4'd0VPRVR,R RVR,R, R R24VPR4'dVPRVR, 24VPR4'dVPRVR, 24VPR4'dVPRVR, 24VPR 4VPPR/4p.pVPR4'd.pVPR 4'dVPR!4VPR"4'dVPR#4VPR$4'dVPR%4VPR&4'dVPR'4V'd$VPR(R)PV4 24VPR*4'dVPR+4VPR,4'dVPR-4VPR.4'dVPR/4VPR04'dVPR14VPR24'dVPR3VR2, 24V'd@VPR44VFpVPR5V 24K VPR 4VPPR6/4p.pVPR74'dVPR84VPR94'dVPR:4VPR;4'dVPR<4VPR=4'dVPR>4VPR?4'dVPR@4VPRA4'dVPRB4VPRC4'dVPRD4V'd5VPRER)PV4 24VPR 4VPRF4e9VRF,'dRGMRHp VPRIV 24VPR 4VPPRJ.4p V 'd<VPRKR)PV RL,4 24VPR 4VPPRMR 4p V 'dKVPRN4VPV RO,4\ V 4RP8dVPRQ4RRPV4#)Sz)Convert facts to GPT-friendly text formatr z Property: r#rjz Location: r"u Price: €rir>r$r~zType: rxz Total area: u m²ryz Living area: rzz Land area: u m² (rZz.2fz ha)r{z Bedrooms: r|z Bathrooms: rz Year built: r%rrrrrrrrz river accesszWater: z, rzForest/woodland presentrOrchardrzPasture/meadowrzIrrigation availablerzSoil: zLand features:z - r&rrrrrrrrr outbuildingsrrrzseparate apartmentzAdditional buildings: rzRenovation neededz Move-in readyz Condition: r'z Amenities: rIr!z Description::NiNiz... )rgetrNrOrJ) rlinespdld land_features water_typesr9bd buildingsstatusr'descs & rto_prompt_text%PropertyFactsExtractor.to_prompt_text}s ::>>' " " LL:djj&9%:; < ::>>*b ) - -f 5 5 LL:djj&>' " " LL:djj&9!%<= > RZZ^^. 3 66/ " " LL6"_"5!67 8 66/ " " LL<?(;'dC D 66. ! ! LL;r.'9!&R-S  LL1$))I2F1GH I LL  66% & 2,./B,C,C(F LL;vh/ 0 LL JJNN;3  LL;tyy3'@&AB C LL zz~~mR0  LL ( LLd $4y3 U#yyr)rrr N)__name__ __module__ __qualname____firstlineno____doc__rr7r+r,r-r.r/r0r1r2r3r4r5rWrh__static_attributes____classdictcell__)rs@rrr s5 $;; - -##J   FFP==~..`$$:(*  DD` ` rrc^V^8dQhR\R\R\\\3,/#)r r r r)rrr)rs"rrrs) # # #3 #4S> #rc8\W4pVP4#)z Main function to extract structured facts from property HTML Args: html_content: Raw HTML from property page url: Property URL Returns: Dictionary with structured property facts )rr7)r r extractors&& rextract_property_factsrts'|9I  ""r__main__zJhttps://www.properstar.com/property-for-sale/france/lot-et-garonne/7837046z Testing Property Facts ExtractorzURL: r]z User-AgentzDMozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15)headerstimeoutzSTRUCTURED JSON:z GPT PROMPT FORMAT:zFailed to fetch page: zF======================================================================)rnrKbs4rtypingrrrrrUrrtrjrequeststest_urlprintrvr^response status_coderSrrsr7rWrhrrrs ,, R R j # z[H ,- (O E(2  \G||HgrBHs"&x}}h?*8==(C   ! h i!" &' h i&&() &x';';&<=>;r